Request: Ollma & Open WebGUI

Jarel_Trenton · May 19, 2024, 10:50am

Ollama - Get up and running with large language models: Download Ollama on macOS

Open WebUI - AI for Chat Bots https://openwebui.com

dragonfire1119 · May 20, 2024, 1:08am

Added:

aeonoea · August 8, 2024, 3:47pm

Can we have GPU support for Intel too?

github.com

intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/ollama_quickstart.md

# Run Ollama with IPEX-LLM on Intel GPU

[ollama/ollama](https://github.com/ollama/ollama) is popular framework designed to build and run language models on a local machine; you can now use the C++ interface of [`ipex-llm`](https://github.com/intel-analytics/ipex-llm) as an accelerated backend for `ollama` running on Intel **GPU** *(e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max)*.

See the demo of running LLaMA2-7B on Intel Arc GPU below.

<table width="100%">
  <tr>
    <td><a href="https://llm-assets.readthedocs.io/en/latest/_images/ollama-linux-arc.mp4"><img src="https://llm-assets.readthedocs.io/en/latest/_images/ollama-linux-arc.png"/></a></td>
  </tr>
  <tr>
    <td align="center">You could also click <a href="https://llm-assets.readthedocs.io/en/latest/_images/ollama-linux-arc.mp4">here</a> to watch the demo video.</td>
  </tr>
</table>

> [!NOTE]
> `ipex-llm[cpp]==2.5.0b20240527` is consistent with [v0.1.34](https://github.com/ollama/ollama/releases/tag/v0.1.34) of ollama.
>
> Our current version is consistent with [v0.1.39](https://github.com/ollama/ollama/releases/tag/v0.1.39) of ollama.

This file has been truncated. show original

edit: there seems to be native support being worked on here, but I don’t quite get what’s going on:

github.com/ollama/ollama

Enabling ollama to run on Intel GPUs with SYCL backend

ollama:main ← zhewang1-intc:rebase_ollama_main

opened 05:44AM - 21 Mar 24 UTC

zhewang1-intc

+614 -31

Hi, I am submitting this pr to enable ollama to run on Intel GPUs with SYCL as t…he backend. This pr was [originally](https://github.com/ollama/ollama/pull/2458) started by @felipeagc who is currently unable to actively participate due to relocation. The original pr had fallen behind the main branch, making it inconvenient for maintainers @mxyng @jmorganca @dhiltgen to review. Therefore, I rebased the latest main branch and opened this new pull request. I have verified that it works correctly on Ubuntu 22.04 with ARC 770 GPU. While I am not very familiar with this project and I welcome any guidance and assistance from the community. Let’s work together to make ollama support Intel GPU platforms. cc:@hshen14 @kevinintel @airmeng UPDATE: works well on windows10 + ARC 770 UPDATE: works well on oneapi-docker-image(oneapi-basekit-Ubuntu22.04) + ARC770