| ▲ | nicoburns 43 minutes ago | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Interesting. You're claiming that the Opus/Sol/Neon class and Fable/Astra/Carbon class models are similar sizes to each other? That would be surprising to me. I've found that while Opus 5.5 is often pretty nice for practical work, Fable (even 5.0) is still significantly smarter on hard tasks and makes far fewer "dumb mistakes" | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | dannyw 36 minutes ago | parent | next [-] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
A Microsoft Azure page briefly claimed Sol runs with less looped transformer passes; and so you could speculate that Sol and Astra might share weights; but have different “loop” factors. That would make a lot of sense from an inference optimisation perspective. The same prefill and decode machines can hold one set of weights, and serve both Astra and Sol requests. At (multi-)terabyte-scale model weights, and when you have to offer not just the latest like Fable 5.2 but still serve 5.1, 5.0, etc; heck I think Opus 4.6 is still around; you really care about overall compute utilisation. Idle GPUs are wasted GPUs; and even over the fastest interlinks you’re not hotswapping multi terabytes of weights. With MoEs, there’s probably stuff you can play around with varying the number of active experts; or other clever optimisations. Maybe Sonnet 5.5 could be just Opus 5.5 with a different router and less active params? Who knows; I’m just theorycrafting now :) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | onlyrealcuzzo 29 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Fable & Opus, Astra & Sol, and Carbon are all thought to be >1T param models - AFAIK, most people think Opus & Sol are ~5T param models and Astra, Fable are 10T+. I think the rumor-mill consensus is that Google's models are significantly smaller while attempting to swing at the same level - but the ratios AFAIK are thought to be around the same - i.e. Neon is somewhere around half as big as Carbon, big enough that a newer checkpoint can make it temporarily a little bit better. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | andai 24 minutes ago | parent | prev [-] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The Bitter Lesson - Richard Sutton (2019) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||