| ▲ | Topfi a day ago | |
Good move given some experienced issues and compaction across the 5.6 range is closer to 5.4 than 5.5, i.e solid and reliable. Will say that 5.6-Sol is a minor bump in my benchmarks in most areas vs 5.5 but a severe regression in a few specific task focused on rearranging trees, addressing merge conflicts, etc. where the model to accomplish the task does not properly adhere to prompts in a way GPT-5 originally managed, not retaining parts of history in the way prompted despite specific instructions not to as that made the final completion easier… I am of the conservative and cautious opinion that no model should be able to run destructive tasks at all, I have seen every model do things that make me concerned enough to maintain that opinion and know my evals can’t catch everything. But for 5.6-Sol specifically, I’d caution everyone to reevaluate how you run the model, maybe take a few more precautions you tend to forgo. It is extremely capable as a reviewer and for extensive tasks, though for the later, the safety net I feel is required to be comfortable limits the utility. The code 5.6-Sol provides also still is a bit harder to parse in reviews. Release strategy wise, feel it’s have been smarter to release only Luna and Sol now, then Terra a few weeks of posttraining later, I simply cannot see a purpose for it in the current form given how well both Luna and Sol scale up and down respectively with reasoning. Two models from a lab at a time is also the limit I feel one can properly assess at a time. | ||