| ▲ | margalabargala 5 hours ago |
| The obnoxious/confusing thing is that models are released individually rather than as a family. Luna < Terra < Sol < Astra, fine. But does Terra 6.1 beat Sol 6? Does Fable 5.1 beat Opus 5.5? If I build something around this Terra 6.1 release, then when Luna 6.2 is released in a month, should I swap? |
|
| ▲ | cdirkx 4 hours ago | parent | next [-] |
| I understand the confusion, but isn't this kind of similar to asking does the iPhone 14 Max beat the iPhone 16 Mini? At least with phones there are hard specs we can use to compare, but in a lot of cases the answer will be "it depends" |
| |
| ▲ | Closi 4 hours ago | parent | next [-] | | At least there is a simple system - the words 'Mini' and 'Max' are intended to refer to the size of the item (that one is miniature, and the other is their maximum size). If they were called iPhone 14 Luna and iPhone 16 Sol then this is less clear. Or on the flipside, if the models were called GPT 6.1 Mini and GPT 6.1 Max, nobody would be arguing about the names being hard to grok. The words 'mini' or 'small' are more descriptive than 'Luna' or 'Terra'. | | |
| ▲ | lukeschlather 2 hours ago | parent [-] | | The problem with that is then you would have GPT 6.1 Max with max thinking which is different from GPT 6.1 Small with max thinking. If you look back the naming problem started around the point that OpenAI started doing reasoning models (o1, 4o) where they had to describe this fork in model behavior that let a smaller model think longer and be smarter. You can come up with naming schemes that don't clash, but there's something to be said for this approach where the thinking level is like you describe and the model name requires you to learn that Astra is the largest model. |
| |
| ▲ | margalabargala 2 hours ago | parent | prev | next [-] | | > does the iPhone 14 Max beat the iPhone 16 Mini? Sure, in a hypothetical world where the latest Max phone is two generations older than the latest Mini phone. In which case now you have to evaluate not only screen size but ALSO specs and component age and software support etc. How badly do you need the bigger screen? Etc If you are comparing the iPhone 16 Max to the iPhone 16 Mini, everything is a lot easier. | |
| ▲ | saagarjha 3 hours ago | parent | prev | next [-] | | > iPhone 16 Mini I wish | |
| ▲ | XCSme 3 hours ago | parent | prev | next [-] | | Yeah, but shouldn't then all 4 tiers be released at once? So we get 6.1 for all Luna/Terra/Sol/Astra? | |
| ▲ | someguyiguess 2 hours ago | parent | prev | next [-] | | No. It’s not similar at all. | |
| ▲ | nailer 4 hours ago | parent | prev [-] | | It's more like if Apple called the iPhone 16 the "iPhone 14 16 edition" | | |
|
|
| ▲ | elxr 4 hours ago | parent | prev [-] |
| Run your own evals then if you have this much of a need to "correctly" choose. That'll make it clear. I'll just stick to mostly Sol on low/medium/high and start experimenting with Terra tomorrow onward to see what it does noticeably better than Luna/Deepseek flash. Any pareto frontier model, especially from the leading labs is guarnateed to be good enough for whatever your expectations are at this point. A quick few day test should be enough for anyone to make their own conclusions. |
| |
| ▲ | margalabargala an hour ago | parent | next [-] | | Right, having products so poorly described that a customer needs to run their own evals to choose well is exactly what is being complained about. | |
| ▲ | 2 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | solyin 3 hours ago | parent | prev | next [-] | | Or I'm going to give up and not use any. Consumers aren't perfectly rational. This isn't like toothpaste, where, despite being overwhelmed at the store, I still need to brush my teeth so ahhhhhh fuckit and just pick a toothpaste out of the 30 available. The alternative is I keep using Google search and think AI is stupid. Or just use Facebook's thing. | | |
| ▲ | 2III7 6 minutes ago | parent [-] | | The frontier toothpaste is the one with bioglass 45S5. Especially if you have sensitive teeth or your teeth are prone to caries. |
| |
| ▲ | nailer 4 hours ago | parent | prev [-] | | > > But does Terra 6.1 beat Sol 6? > Run your own evals then if you have this much of a need to "correctly" choose. No. OpenAI want my money, it's on them to communicate properly. | | |
| ▲ | nharada 3 hours ago | parent | next [-] | | What would you want to see instead? Not sure about you but my needs are specific enough that I don't think OAI has evals for them, and I wouldn't want to give them that data even if they wanted to run it for me. | | | |
| ▲ | ssl-3 4 hours ago | parent | prev [-] | | How does one more-properly articulate "it depends"? | | |
|
|
