Remix.run Logo
YawningAngel a day ago

My experience of Sol is that it's very instrumentally competent but is a complete terrorist that will cheerfully do unhinged things that "technically" solve a problem. For example, I asked it to verify something in our integration tests and it responded by grabbing kafka details out of config and trying to use them. When I pointed out that this was not it, it did the same thing with a database client instead. Conversely, I'm pretty sure Fable or even Opus would have done something consistent with what the rest of the codebase does (consume an API or point out that there's no API to access in this case).

I don't have Fable at work but I'd probably use it for actual code if I did because not having to spend time handholding the model on this stuff and getting useful code first try is very useful

edg5000 7 hours ago | parent [-]

What I do notice is that when it doesn't have the info it needs, or when I don't tell it that I'll handle part of the process, it will indeed do everything to get the job done. Also saw this but find this a great quality, because I never ask it to do anything that would involve unhinged things. Sol and other models as well have just gotten so damn good. They can get just about anything done and usally do it well too. Maybe I was inexperienced in prompting, but I remember doing much, much more handholding with late 2025 LLMs.