I'm finding qwen 27b is comparable to sonnet but my self hosting has about 5 more 9s than whatever Anthropic's vibe coding. I also don't have to worry about the quality of the model I'm being served from day to day.

Probably the most damning fact about LLMs is just how poorly written their parent companies' systems are.

But do you actually treat LLMs as glorified autocomplete or treat them as puzzle solvers where you give them difficult tasks beyond your own intellect?

Recently I wrote a data transformation pipeline and I added a note that the whole pipeline should be idempotent. I asked Claude to prove it or find a counterexample. It found one after 25 minutes of thinking; I reasonably estimate that it would take me far longer, perhaps one whole day. I couldn’t care less about using Claude to type code I already knew.

    > Probably the most damning fact about LLMs is just how poorly written their parent companies' systems are

During this process of investigation, I checked the CC repo and noticed they had 5000+ issues open. Out of curiosity, I skimmed through them and many point to regressions, real bugs, simple changes, etc. Maybe they have some internal tracker they are using, but you would think that a company with functionally unlimited tokens and access to the best models would be able to use those tokens to get their own house in order.

My sense now is that there is a need for the industry to create a lot of hype right now so we see showmanship like the kernel compiler and the agent swarms building a semi-functional browser, etc....yet their own tooling has not fully implemented their own protocol (MCP) correctly. They need all of us to believe that these agents are more capable than they actually are; the more piles of tangled code you write and the more discipline you cede to their LLMs, the more dependent you are on those LLMs to even know what the code is doing. At some point, teams become incapable of teasing the code apart anymore because no one will understand it.

Peeking at the issues in the repos and seeing big gaps in functionality like Codex's missing support for MCP prompts and resources is like looking behind the curtain at reality.

I've tried a few models and some are decent, including Qwens models. I've tried a few harnesses like Roo Code in VSCode to put things together that in theory emulate the experience I get from VSCode + Claude or Copilot, but I generally find the experience extremely limited and frustrating.

How have you set things up to have a good experience?

People keep saying this and idk what I'm doing wrong. Using q8_0 on all the latest and greatest local models and they just don't come close to sonnet.

I've tried different harnesses, building my own etc.

They are reasonably close to haiku? Maybe?

Just to make one obvious critique your costs per token are probably about 1000x higher than the ones they provide.

I'm pretty sympathetic to Anthropic/OpenAI just because they are scaling a pretty new technology by 10x every year. It is too bad Google isn't trying to compete on coding models though, I feel like they'd do way better on the infra and stability side.

What do you run it on? And even then, I'm guessing your tokens per second are not great?

> Probably the most damning fact about LLMs is just how poorly written their parent companies' systems are.

This seems like a popular take, but I think it's the other way around. Them dogfooding cc with cc is proof that it can work, and that "code quality" doesn't really matter in the end.

Before cc claude.ai (equivalent of chatgpt) was meh. They were behind in features, behind in users, behind in mindshare. cc took them from "weirdos who use AI for coding" to "wait, you're NOT using cc? you freak" in ~1 year. And cc is a very big part of them reaching 1-2B$ monthly revenue.

Yes, it's buggy. Yes, the code is a mess (as per the leak, etc). But they're also the most used coding harness. And, on a technical side, having had cc as early as they did, helped them immensely on having users, having real-usage data, real-usage signals and so on. They trained the models on that data, and trained the models in sync with the harness. And it shows, their models are consistently the highest ranked both on benchmarks and on "vibes" from coders. Had they not have that, they would have lacked that real-world data.

And if you look at the competition it's even more clear. Goog is kidna nowhere with their gemini-cli, is all over the place with their antigravity-ex-windsurf, and while having really good generalist models, the general mindshare is just not there for coding. Same for oAI. They have an open-source, rust-based, "solid" cli, they have solid models (esp in code review, planning, architecture, bug fixing, etc) but they are not #1. Claude is with their cc.

So yeah, I think it's really the other way around. Having a vibe-coded, buggy, bad code solution, but being the first to have it, the first to push it, and the first to keep iterating on it is really what sets them apart. Food for thought on the future, and where coding is headed.

QWEN3.5-Next-Coder does wonders. It's drawbacks are time to first token is 30 seconds to load the model and OpenCode has an unsolved timeout issue on this load, but otherwise once it's warmed up, it's entirely serviceable.

I've got a AMD395+ with 128GB, so running a ~46GB model gives me about 85k tokens, which gives me easily copy/paste/find/replace behavior; it mocks up new components; it can wire in some functionality, but that's usually at it's limits and requires more debugging.

I've been looking at how to schedule it using systemd to keep a wiki up to date with a long loaded project and breaks the "blank page" issue with extending behaviors in a side project.

I understand some of these larger models can do things faster and smarter, but I don't see how they can implement novel functionality required for the type of app I'm concerned with. If I just wanted to make endless CRUD or TODO apps, I'm betting I could figure out a loop that's mostly hands off.