new | show | ask | jobs Github

wkat4242 7 days ago

But there's much less that works with that. OpenWebUI for example.

▲

vntok 6 days ago | parent [-]

Open WebUI works perfectly fine with llama.cpp though.

They have very detailed quick start docs on it: https://docs.openwebui.com/getting-started/quick-start/start...

▲

wkat4242 6 days ago | parent [-]

Oh thanks I didn't know that :O

I do also need an API server though. The one built into OpenWebUI is no good because it always reloads the model if you use it first from the web console and then run an API call using the same model (like literally the same model from the workspace). Very weird but I avoid it for that reason.

	▲	rpdillon 6 days ago \| parent [-]
		llama.cpp is what you want. It offers both a web UI and an API on the same port. I use llama.cpp's webui with gpt-oss-20b, and I also leverage it as an OpenAI-compatible server with gptel for Emacs. Very good product.