[flagged]

dang 7 days ago | parent | next [-]

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

https://news.ycombinator.com/newsguidelines.html

	▲	7 days ago \| parent [-]
		[deleted]

▲

api 7 days ago | parent | prev | next [-]

> Repackaging existing software while literally adding no useful functionality was always their gig.

Developers continue to be blind to usability and UI/UX. Ollama lets you just install it, just install models, and go. The only other thing really like that is LM-Studio.

It's not surprising that the people behind it are Docker people. Yes you can do everything Docker does with Linux kernel and shell commands, but do you want to?

Making software usable is often many orders of magnitude more work than making software work.

▲

otabdeveloper4 7 days ago | parent [-]

> Ollama lets you just install it, just install models, and go.

So does the original llama.cpp. And you won't have to deal with mislabeled models and insane defaults out of the box.

▲

lxgr 6 days ago | parent [-]

Can it easily run as a server process in the background? To me, not having to load the LLM into memory for every single interaction is a big win of Ollama.

▲

otabdeveloper4 6 days ago | parent [-]

Yes, of course it can.

	▲	lxgr 6 days ago \| parent [-]
		I wouldn't consider that a given at all, but apparently there's indeed `llama-server` which looks promising! Then the only thing that's missing seems to be a canonical way for clients to instantiate that, ideally in some OS-native way (systemd, launchcd etc.), and a canonical port that they can connect to.

▲

llmtosser 7 days ago | parent | prev | next [-]

This is not true.

No inference engine does all of:

- Model switching

- Unload after idle

- Dynamic layer offload to CPU to avoid OOM

	▲	ekianjo 7 days ago \| parent [-]
		this can be added to llama.cpp with llama.swap currently so even without Ollama you are not far off

▲

mchiang 7 days ago | parent | prev | next [-]

sorry that you feel the way you feel. :(

I'm not sure which package we use that is triggering this. My guess is llama.cpp based on what I see on social? Ollama has long shifted to using our own engine. We do use llama.cpp for legacy and backwards compatibility. I want to be clear it's not a knock on the llama.cpp project either.

There are certain features we want to build into Ollama, and we want to be opinionated on the experience we want to build.

Have you supported our past gigs before? Why not be more happy and optimistic in seeing everyone build their dreams (success or not).

If you go build a project of your dreams, I'd be supportive of it too.

	▲	Maxious 7 days ago \| parent [-]
		> Have you supported our past gigs before? Docker Desktop? One of the most memorable private equity rugpulls in developer tooling? Fool me once shame on you, fool me twice shame on me

▲

dangoodmanUT 7 days ago | parent | prev [-]

Yes everyone should just write cpp to call local LLMs obviously

▲

otabdeveloper4 7 days ago | parent [-]

Yes, but llama.cpp already comes with a ready-made OpenAI-compatible inference server.

	▲	reverius42 6 days ago \| parent [-]
		I think people are getting hung up on the "llama.cpp" name and thinking they need to write C++ code to use it. llama.cpp isn't (just) a C++ library/codebase -- it's a CLI application, server application (llama-server), etc.