Mental model:

A LLM is basically a program runtime. Code in -> output. There's a P(correct output|program), and better your model or the program, the higher it is. Even a bad model can produce the right output if you feed it the right program -- the hardest output is easy if the program is just "here's the output I want you to produce, parrot it verbatim". The key is being able to search for a program that has the highest marginal P(success) efficiently.