Remix.run Logo
magicalhippo 4 hours ago

Here[1] is the offending pull request. The llama.cpp devs are trying to refactor the code, so the "engine" bits are shared between the current server and client applications.

Georgi himself noted it should be possible to replace the transport layer from HTTP to say an internal message queue, and the main developer of the PR said the HTTP bit is already abstracted and intended to include a non-HTTP transport example, so seems clear to me it's not intended to be HTTP-only.

[1]: https://github.com/ggml-org/llama.cpp/pull/24948

trilogic 3 hours ago | parent [-]

How is adding compulsory HTTP to CLI necessary for refactoring the code?

You realize that now the CLI needs http permissions to load a model locally. Still confused why, the server already does that by all means.

First they changed the remote downloading weights in server. Then they approved and merged claude code snippets. Now mandatory monitoring.

Everyone has a price