Remix.run Logo
▲ solenoid0937 4 hours ago

Hot take, none of the models are getting "nerfed", people are just getting used to the new level of intelligence.

▲solfox 3 hours ago | parent | next [-]

No, whether or not it's intentional, maybe can be debated. But there's definitely an experience of a model losing horsepower quickly after launch.

▲jyoung8607 3 hours ago | parent | next [-]

Has there ever been any measurement of this, of any sort? Honest question. I frequently see a plural of anecdotes to that effect, but I've not seen a concrete statement of fact or measurement that could be scrutinized or tested in any way.

If so, please share. This should be measurable, and I'm glad this project is measuring it.

Answers in the form of additional anecdotes, stated with even greater passion but still lacking a statement that could be tested and falsified, would validate my exact concern.

▲Craighead 3 hours ago | parent | prev | next [-]

prove it

▲bradfa 3 hours ago | parent [-]

Literally the point of the linked GitHub repo.

▲nimchimpsky 3 hours ago | parent | prev | next [-]

[dead]

▲nba456_ 3 hours ago | parent | prev [-]

No there isn't.

▲voiceeh 3 hours ago | parent | next [-]

You mean to write YOU haven't experienced this. Many have indeed experienced this.

▲doginasuit 3 hours ago | parent | next [-]

I'm not sure "experienced" is carrying much weight here. This is why there are benchmarks, anecdote doesn't mean very much.

▲nba456_ 3 hours ago | parent | prev [-]

No you didn't.

▲omani 3 hours ago | parent | prev | next [-]

who is paying you to say that?

▲nba456_ 3 hours ago | parent [-]

Mr. Dario himself

▲ 3 hours ago | parent | prev | next [-]
[deleted]
▲AnimalMuppet 3 hours ago | parent | prev [-]

The claim was that there's an experience of a model losing power. Your claim amounts to "No, you are not experiencing what you say". That's quite a claim for you to make with no data and no argument.

▲jascha_eng 3 hours ago | parent | prev | next [-]

Yeh it's absurd that people claim this all the time. It's some crazy conspiracy theory and when you ask for examples nothing ever shows up.

It would be economical suicide from anthropic and OpenAI to actually need models intentionally.

But hey I guess it's hard with technology that truly seems like magic. People say if you'd bring electricity to the middle ages you'd be called a witch and burned. The same is happening to the model labs here because they are bringing tech that the world isn't ready for yet.

▲wccrawford 3 hours ago | parent | prev | next [-]

I was just wondering if, like certain processors, bugs get fixed and the speed goes down. Like, they find it's doing things it shouldn't, restrict it, and harm the throughput.

▲raincole 3 hours ago | parent | prev | next [-]

It's not a hot take at all. Every benchmark shows that.

▲dude250711 3 hours ago | parent | prev | next [-]

Suuuuure...

▲empath75 3 hours ago | parent | prev [-]

Yeah people push the models to the limits of what they are capable of almost instantly.