> The concept of infinity claims that the dog eventually becomes Shakespeare.

The ability to reproduce an exact copy of hamlet does not make one Shakespeare. A monkey on a typewriter may very well generate Shakespeare eventually, but it wouldn't understand Shakespeare then any more than it could immediately. Likewise a dog may put together some string of text that includes a derivation of calculus, but at no time will it be able to apply that derivation to solve mathematical problems.

People seem to think entropy can be overcome with proper focus. Thats why we have things like "effective altruism", the idea that you can ignore all the harm you do on the way to some big grand altruistic act, as if the shattered glass can be reassembled if you just collect enough reverse entropy.

It's a line of reasoning meant to shut off empathy to the here and now. And while it sounds good, along the lines of Baywatch: If you're jumping into a live saving situation and you have to choose between further harming your victim and you being harmed, you choose your victim because without you to save both of you, it's fatal; the difference is indirectly or directly pushing your victim into the water then claiming you're altruistically going to save them at a later date.

It's just delusions to keep moving forware.

> Mythos and other models are not brute-forcing passwords (and with this analogy passwords, ie. systems are the same).

I am not talking about literally bruteforcing passwords (although LLMs are being used for that, too), but bruteforcing passwords and solving verifiable domain tasks have quite a few similarities, especially when considering rule-based and probabilistic bruteforce methods.

> We're not talking about dogs, but LLM systems.

Well, clearly dogs are not LLM systems. It is an analogy. If there is an important point on your mind that makes the analogy break down, feel free to spell it out.

> Mythos is not exploring entire solution space either.

Yes, but weaker models do not find the solution right away, so they need to try more often. But if they only try the same thing every time, they will never succeed, so we need some kind of guarantee that they try something different every time.

> Usually looping is solved by repetition/frequency/presence/n-gram penalties/DRY/min-p sampling, not temperature but we're not talking about small models that have those classes of issues here.

Those might help to reduce looping (at the cost of biasing the generation), but to guarantee that a model can generate all possible generations, we need non-zero probabilities for all tokens, not lower probabilities for likely tokens.