I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.

▲

madushan1000 4 hours ago | parent | next [-]

Two days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.

	▲	walrus01 4 hours ago \| parent \| next [-]
		the full set of llama.cpp binaries builds in under 5 minutes with an unmodified build workflow straight from their github page on a literally ten year old dual xeon.
	▲	gnull 4 hours ago \| parent \| prev [-]
		It also even easier to get working and integrate into your system in a sustainable manner with NixOS. Do it by hand or throw an LLM at it, it will get you a declarative patch for your NixOS config that brings llama-cpp into your config that you can review and add under version control (no random `make install` build artifacts contaminating your system, no wondering "what was it that I ran? what are all these files? how do I do the same with a newer version?" a couple months later). There's also likely some build cache where Nixoids have already build what you want. I had a great experience with llama-cpp with Nvidia backend on NixOS. (Sorry for being that guy.)

▲

numpad0 3 hours ago | parent | prev | next [-]

I don't know how it works, but does Vulkan not work on Arc?

▲

cptskippy 2 hours ago | parent | prev | next [-]

Does the A770 use the Xe driver? If so then it might work with the scripts that I've been using to build llama.cpp with SYCL support for the Arc Pro B70.

https://github.com/cptskippy/battlemage-llm-gateway

It's designed so that you can re-run the scripts to pull the latest updates. When Muse Glimmer was released the other day I just ran the 02 script to build the latest version of llama.cpp with support for it.

▲

OpenArcBob 2 hours ago | parent | prev [-]

[dead]