Remix.run Logo
julianlam 8 hours ago

I think it's interesting that people write off open weight models because they're "a few months behind" proprietary models.

I know LLMs move at the speed of light (especially these past few quarters), but if Opus and GPT "a few months ago" were really like open weight models, then there's really no reason to not switch, especially for those who were using these models a few months ago.

Your codebase didn't change, so use the open weight model. Don't move the goalposts.

kgeist 7 hours ago | parent | next [-]

Every new proprietary model is "groundbreaking" and "look, it just solved task X that no other model could solve," only to be referred to as "that crappy previous-generation model" a month later.

So yeah, I'm totally fine using Kimi-2.7, GLM-5.2 or Deepseek-v4. I think we've already hit the ceiling and most improvements now seem to be from harness improvements and slightly better RL to improve reasoning/tool calling.

jbverschoor 5 hours ago | parent | next [-]

Not only that, but to me it seems that after a week the intelligence is being downscaled or routed. Maybe because of lack of capacity

matheusmoreira 5 hours ago | parent | prev | next [-]

There's at least the possibility that they intentionally degrade the models as time passes. We can't really verify that we're getting what we're paying for all of the time. All the more reason to invest in local inference.

inigyou 4 hours ago | parent | next [-]

What if the new model is exactly as good as the last model on launch day but better than the last model was on the new model's launch day because it was degraded? Every single time?

foo42 an hour ago | parent | next [-]

Makes me think of [shepherd tones](Shepard tone - Wikipedia https://share.google/xooRbF7wIIhcsTt2J) which sounds like they're rising in pitch indefinitely

no-name-here 27 minutes ago | parent | prev | next [-]

There are lots of benchmarks to compare the absolute values of different models on the same scale (as opposed to vibes, etc.).

matheusmoreira 2 hours ago | parent | prev [-]

The thought has definitely crossed my mind. I don't think it's true because there's definitely an improvement when new models are released.

Maybe the truth is the newest models aren't actually as impressive as we thought. Maybe our perception of progress is being manipulated via months of gradual, silent and unverifiable degradation.

manyatoms 4 hours ago | parent | prev | next [-]

Unless what you're getting is really explicitly spelled out in a contract, you should flatly assume that they're doing whatever they like whenever they like.

OtomotO 3 hours ago | parent [-]

Even if it's in the contract, but can't be verified.

LPisGood 3 hours ago | parent | prev | next [-]

People talk about this a lot. What I have never seen is a discussion of methods they might employ to degrade the models.

Let’s say I’m a bad faith LLM operator, and I want to degrade my model so the next release looks better and people want to switch to the more expensive one. How would I do that?

nessex 2 hours ago | parent | next [-]

They would quantize the model. That'd make it cheaper to run, and have slightly worse output but it would still generate outputs with a similar feel, derived from a compressed version of the same knowledge base etc.

They wouldn't even need to do this uniformly, quantized versions of the model could be routed only a subset of the requests. They could do this to nerf the old model, or more likely just to give themselves more hardware to run the new one on by handling more requests on less hardware. Or to handle increased request volume as traffic ramps up faster than hardware can be provisioned.

Playing with local models at various quants, the degradation can be hard to spot. Sometimes it's only noticeable in aggregate. And even then, you never really know if you just got unlucky with a bad response due to RNG.

I've had Opus 4.6 fall into some weirdly incoherent loops that I rarely see from even Sonnet, that felt like the kind of thing I got frequently with Qwen3.5 9B on local. And the above applies... Was that just bad RNG? Or was my request to Opus routed to some lower quality variant? There's no great way for me to tell for any given request, nor any way to guarantee Anthropic _didn't_ do that.

OccamsMirror an hour ago | parent [-]

I have had the same experiences you've had with 4.6 and it was ever since they brought out 4.7. It's fairly obvious they're doing something like you've said here.

nessex 31 minutes ago | parent [-]

Forgot to mention, but it was after the 4.7 release when I was still using 4.6 that I saw those loops too... Before that, 4.6 had been a pretty seamless experience.

maybe_pablo 2 hours ago | parent | prev | next [-]

Weight quantization, n-expert capping, routing to smaller model, context window truncation, aggressive sampling constraints, lossy speculative decoding and probably more.

alfiedotwtf 12 minutes ago | parent [-]

I'm pretty sure you could do n-expert capping on any MoE model with only a handful lines of changes to ik_llama.cpp, but yeah... my bet is the have various quantisations and run the lower ones at peak (along with different system prompts i.e we're GPU-bound right now. Get to the point with less chatter)

Tepix 2 hours ago | parent | prev [-]

Use quantisation.

taytus 5 hours ago | parent | prev [-]

At current prices, and considering these OS Models' performance, investing in local inference sounds like a bad idea.

matheusmoreira 5 hours ago | parent | next [-]

Current prices are insane but at this point I'm starting to feel like it's an existential issue. I'm not a US citizen. At any point the USA could come up with some arbitrary export controls. Not having a computer capable of running at least Qwen is starting to actually seem risky to me.

At least it's going to be usable as a very high end gaming PC.

alfiedotwtf 11 minutes ago | parent | next [-]

> At any point the USA could come up with some arbitrary export controls

lol his already happened with Fable!

awakeasleep 4 hours ago | parent | prev [-]

Why would you buy and build everything before the low probability catastrophe strikes, though? You don’t get any benefit from switching early and you pay a big opportunity cost.

Lapel2742 3 hours ago | parent | next [-]

> low probability catastrophe

There is also a low probability that someone enters peace negotiations solely to threaten the negotiators with death, yet here we are. With these guys it is: Better safe than sorry.

inigyou 4 hours ago | parent | prev | next [-]

because as soon as it strikes computer hardware will be completely unavailable to buy?

CamperBob2 3 hours ago | parent [-]

Also, there's a nontrivial learning curve involved in running your own inference server, once you move past the casual-goofing-around-with-llama-server stage. If you care about not being a sharecropper on Sam's or Dario's plantation, you should consider learning the ropes. Even if you don't put these skills to immediate use in your day job.

I didn't appreciate this until I started down that road myself.

matheusmoreira 2 hours ago | parent [-]

> If you care about not being a sharecropper on Sam's or Dario's plantation

Couldn't have put it better myself. That's what all this comes down to. Owning the hardware, owning the inference. Not perpetually renting them out on a meter like in the dystopian future they're envisioning.

OtomotO 3 hours ago | parent | prev [-]

Because you will not be the only one struggling to get the hardware in the "unlikely" case the POTUS blurts out another fart.

jrm4 5 hours ago | parent | prev [-]

At current "proprietary inference company behavior," investing in local inference sounds like the exceedingly far more rational option.

Long term predictability ought to far outweigh a few more cycles of performance.

realusername 5 hours ago | parent | prev | next [-]

There's also a lot of benchmark trickery going on, it's becoming harder to see how the latest models really improved.

The top models also seem to have inconsistent performance depending on the time of day and how far we are from the next release.

bonesss 4 hours ago | parent | next [-]

I’m an LLM fan, but from an engineering perspective the idea of building atop services that palpably fluctuate in capacity, performance, and capability is nutty.

Even with minor automation I feel like I can watch OpenAI and Anthropic engineers fiddling in real-time. Tuesdays behaviour changes by Thursday, 10AMs production isn’t possible at 11:30AM. Nutty.

targafarian 4 hours ago | parent [-]

I chilled significantly on using Google for anything to do with business due to API (and offering) stability. (Still use Google for personal things.) But AI models seem orders of magnitude more fluid, so to my risk-averse eye, they're nothing I'd base my own business on.

Barbing 4 hours ago | parent | prev [-]

Interesting, Claude might be doing better since I last checked:

https://marginlab.ai/trackers/claude-code-historical-perform...

There were at least a couple of these degradation trackers.

fsuts 2 hours ago | parent | prev | next [-]

Agreed

4fffs 6 hours ago | parent | prev [-]

Correct. Anything else is pure marketing and you have fallen for it.

Aurornis 5 hours ago | parent | prev | next [-]

> I think it's interesting that people write off open weight models because they're "a few months behind" proprietary models

I experiment a lot with the open models and I’m getting tired of this trope. I’m not yet convinced that even the best open weight models are equal to Opus from “a few months” ago.

I know what the benchmarks say. I had higher hopes. My real experience just doesn’t match the benchmarks.

I also do a lot of work that even Opus 4.8 struggles with. When even the cutting edge LLMs aren’t all the way there yet, my motivation to switch to something even further behind just isn’t there.

CamperBob2 4 hours ago | parent | next [-]

Have you found anything specific that the full-precision quant of GLM 5.2 can't do that Opus 4.8 can? I haven't, so far.

5.2 lives up to the hype. I don't find it to be the best at anything except coding. But for coding... yeah, it lives up to the hype. Not quite Opus 4.8-level, but I would feel comfortable comparing it to 4.5, at least if it had vision capabilities.

iot_devs 3 hours ago | parent | prev | next [-]

I would love if you could make some examples

OtomotO 3 hours ago | parent | prev [-]

> My real experience just doesn’t match the benchmarks.

That's exactly the problem I have... with Anthropic and "Open""AI"

13 minutes ago | parent | prev | next [-]
[deleted]
dwoosley 6 hours ago | parent | prev | next [-]

The only reason I'm on HN right now reading this post is because the Anthropic's API is down... so there's another point for self hosted.

827a 4 hours ago | parent | prev | next [-]

Intelligence is maybe a few months behind. But cost sadly is further behind. GLM-5.2 has a deceptively high cost during day-to-day usage for e.g. coding because 1) it has to think a ton more than GPT-5.5/Opus-4.8 to get to competitive results; 2) many providers are still figuring out caching; and 3) API pricing for Codex/Claude can be as high as 40x more than subscription pricing, which distorts the market.

itwaswatson 5 hours ago | parent | prev | next [-]

We have a provider with Deepseek V4 flash at our work. It can handle 95% of the "actually functional" workload at a tenth of the cost. I still pull up beefier ones sometimes, but that's after some consideration.

The moat is so flat, it only gives +1 food and +1 production. +1 gold with a road.

taormina 7 hours ago | parent | prev | next [-]

For that matter, the new models are shit. If I’m using Opus 4.6 anyway to get anything actually done, then great, we’re actually entirely caught up then.

Gigachad 6 hours ago | parent | prev | next [-]

The reason for me is work pays for Github Copilot which doesn't have these open modals.

4 hours ago | parent | prev | next [-]
[deleted]
TacticalCoder 7 hours ago | parent | prev | next [-]

> I think it's interesting that people write off open weight models because they're "a few months behind" proprietary models.

The really interesting thing is that it's typically those very same accounts who were explaining, a few months ago, that thanks to their commercial model they were gaining so much time and producing so much fantastic code.

A few months passes and suddenly the open-source model have caught up with the models that were gaining them so much time and that produced amazing code (in production everywhere for sure btw) but... It's impossible to work with these models.

Rinse and repeat.

The current models, according to them, are basically AGI and they can go fishing while paid subscriptions solve the world's problems.

But when it six months there shall be new closed, pricey, models and when the open ones shall have reach the level of Fable, we'll hear how it's impossible to work in late 2026 on a model that is "only at the level of Fable".

These people should have been snake-oil salesmen (and it could be what they actually are).

nemomarx 6 hours ago | parent [-]

My most charitable interpretation that there's some honeymoon effect for each release, and people genuinely feel very productive and useful for 2-3 months. By the time the next big model release happens they've seen some issues or run into something that makes them feel like the new model will fix all that and improve their flow so much, etc.

Not unusual in the tech space, but this has been basically constantly happening for two years now? I can't imagine the improvements are more than incremental at this point.

windexh8er 4 hours ago | parent [-]

They are generally referred to as the Kool-Aid drinkers. There's always something holding them back from open models. It's no different than the argument in the article. I've been daily driving Linux for well over 20 years at this point and while things have gotten easier they haven't gotten that much easier. There's always been a distro that's focused on new users or ease of use. I used to take for granted the Linux distro ecosystem but now worry how Microsoft, Apple and others will continue to try and legislate compute into a corner. I can appreciate good engineering, but when I look at OS X and Windows they're both failing end users in different ways.

Just like the OS ecosystem I think we'll see a similar trajectory with OAI, Anthropic and Google but on a much accelerated time scale. I think the lobbying has begun to lock in their fate for revenue - because none of them give a shit about their users. I do hope, however, that Anthropic continues to over rotate and continue to gimp their models into uselessness. I just asked Opus 4.8 the other day to look at some code as an adversary and summarize areas that should be addressed. Nothing specific and it shut down the conversation. However starting a new prompt and prodding the model from a different angle yielded the results I asked for directly. Pick a lane. Or, don't and continue to lose industry respect and consideration.

tonfreed 6 hours ago | parent | prev | next [-]

Even just one of the smaller models is good enough for the grunt work I use them for 90% of the time. Currently doing most of my home hobby projects with OpenCode Go and Qwen 3.7 Plus, it's not great at diagnosing issues in the code, but if I can clearly articulate a test suite or boilerplate refactoring it works fine.

moomoo11 4 hours ago | parent | prev [-]

ok but your competition using the latest models has an advantage

not all of us are doing noob shit lol