Yagil, thanks for the credits! Lots of fun to try running this locally. I don't understand the negativity in some of the other comments.

This is one of the better agent harnesses I've seen for inspecting reasoning chains, which is super useful for me. Sometimes reading the reasoning is better for my needs than reading the response. I appreciate how transparent that is in contrast with Claude Code/Codex/etc.

One question — I saw you mentioned negotiating ZDR with your "providers" — are you hosting the models yourselves, or is another entity hosting them? If another entity, which group?

Can I use my z.ai coding plan API key?

Thanks! I have sent a mail :-D

It's dogshit. It makes the same low quality kindergarten boilerplates the others do - and that's being generous.

Nowhere near ready for production

i agree.

i believe that for most people on the street, for most tasks, a Chat GPT 3.5 era LLM is sufficient enough. sprinkle in tool calling and other things, and that becomes enough. if you can prioritize that level of a model on-device (baking it in etc), then you can bifurcate AI users between those unwilling to pay and those who are willing to pay A LOT for frontier model performance.

This question hinges on whether model advancement plateaus enough for machine sized models to compare to frontier performance. If it does, the answer is yes. If it doesn’t, the answer is no

Neural machines were always going to be an alternative computing paradigm to von Neumann machines. Had it not been for Minsky we would arguably have gotten to a point where they're useful sooner. But why do you say that as if it's a small thing?

I have thought this for a while. Computing 1.0 meant that we needed to learn the computer’s language to interact with the computer fully. Computing 2.0 is that now the computer has learned our language instead.

Why wait? People are already doing their work on OpenAI and Anthropic's servers, Apple Intelligence servers could quickly subsume any "local" model work that you want to do.

That way everyone has access, even with older devices, and it's a subscription! Then Apple can tie their APIs into the ecosystem you love at a flat cost you can afford. No need to support local model integration in the first place, problem solved.

> Does the LLM become another interface to computing?

It already is.

or the other way where primary interfaces ppl use computing arent apple devices like laptops and phones.

based on their apple intelligence demos they are optimizing their products for their core demo of 55-95 year old boomers who talk out loud to think and read every page of the nytimes. you are miles away from the US consumer product experience here.

There’s actually not a lot of good harnesses that aren’t slopped together python or JS codebases, which are actually model-agnostic and don’t do some really silly things like bloating the context, too much compaction etc.

There is no way I’m running a python or JS agent which is probably vibe coded, it’s just too much risk from a security and supply chain standpoint.

Use Unsloth Studio, it's actually open source and I trust Unsloth via their quantized models a lot more than LM Studio.

Yup, it's the main reason I don't use LM studio more. I only use it to try out new models/quants, then use llama.cpp directly to host them. LM Studio also doesn't do stuff like audio input and often has bugs that pure llama.cpp doesn't so it can be a net negative for certain use cases.

Unsloth Studio is open source, run by the same Unsloth that produces some of the best quants in the business, I'd advise people to switch.

I am aware of it, and I dabble with Unsloth Studio and use the llama-server approach.

I would obviously prefer an open source, open weights stack.

But I guess a paradox is that as long as there are open source options I could use, a solid agentic environment that I can use with my own open weights is something I might pay for, in a similar sort of way to paying for a Mac when I could use only Linux.

If someone wanted to make their entire income from, say, making the BBEdit of LLM harnesses, that would be a viable strategy. Sooner or later people need to make an income somewhere. My own feeling is that Apple should acquire LM Studio, but if they said "this is $X per year" I might consider it, given the attention to detail.

Yeah, we already have many open-source agent systems. If you prefer a UI, OpenCode itself has a beta desktop app.

I don't think we need closed-source developer tools, especially ones where they might restrict access if they decide to start charging for them later.

Does anyone know about their stack - is it a native app? It's fairly well designed for what it is in terms of desktop apps.

And? Is that a scandal?

Ultimately, the onus at every VC backed local LLM startup is to launch a cloud based offering, because that's the only potential path in sight for venture scale returns.

Built to work with lmstudio, one of the leading easy to use local model servers. LMStudio is the closest to plug-and-play without sacrificing play that I've seen; a harness that works well with it is nothing to sniff at. Its not earth shattering either.

If you want to use local models, it's more ergonomic than fussing with GGUFs or using LM Studio as a server host and setting up the link to an agent yourself. Although, the model selector is the same as with LM Studio itself which can be overwhelming if you don't know what to look for.

It’s a harness + UI with really confusing messaging. Made for open models ... why specifically? Zero data retention ... well duh, it’s a client????

It does! We negotiated ZDR with our providers. We consider that a condition to make things available to our users: in this case cloud inference and web search/extract.

(I’m the founder of LM Studio)