> The first attempt at recovering the key was wrong in a very specific way; it produced a working key and the signature check passed, but a hash the binary computes as an integrity check didn't match. In my experience, most models would have called it done and left it at that, but Qwen 3.8 27B didn't do that. Instead, it highlighted the mismatch, went back to the drawing board, and kept going until the value matched byte for byte.

This seems to be a pattern in the more recently released models that I think accounts for an increase in the quality of their work. They are very persistent in verifying that their work is actually correct, so even if they're not as "smart" as bigger models that get it right the first time, they have the ability to follow through to ensure that the work is actually done.

Yeah, about a year ago the labs figured out that effective intelligence is a function of persistence as much as anything else. So the models started getting scary persistent late last year, and the trend has continued. There was another jump a few months ago.

Any idea how being persistent is trained? I've noticed that telling an LLM that it needs to think some more sometimes produces better results, but the claim here is that "they are very persistent" and "...kept going...".

Well, it seems that Linus doesn't use those:

> And this was a debug session from hell, enormously helped by an AI doing much of the grunt-work.

> I'd like to call it my tireless helper, but the AI several times stated flat out that this was impossible and unsolvable and that we should just write a report about it.

> I suspect those things have been trained by people who may not be quite as stubborn as I am.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

I believe this is part of the complaints of new models taking longer/requiring higher spend - they go the extra mile on verification, regardless of whether their change is correct already or not. So on problems that an earlier model one-shotted an answer to and did some lighter verification, the newer models might take longer to come back to the user due to running all the tests for your software they could find.

Your explanation makes sense, but I don’t know how you can make a sweeping generalization about more recently released models without qualifying it.

This is says more about humans tendency to pattern match than anything else.

X works better than Y only is only a useful observation if we are using the same X and Y in a similar context, with similar parameters. Kind of goes out the window without it and I think this is part of why people have such vastly different opinions about the same technologies. We’re all talking past each other.