It's the attention mechanism at work, along with a fair bit of Internet one-up-manship. The LLM has ingested all of the text on the Internet, as well as Github code repositories, pull requests, StackOverflow posts, code reviews, mailing lists, etc. In a number of those content sources, there will be people saying "Actually, if you go into the details of..." or "If you look at the intricacies of the problem" or "If you understood the problem deeply" followed by a very deep, expert-level explication of exactly what you should've done differently. You want the model to use the code in the correction, not the one in the original StackOverflow question.

Same reason that "Pretend you are an MIT professor" or "You are a leading Python expert" or similar works in prompts. It tells the model to pay attention to the part of the corpus that has those terms, weighting them more highly than all the other programming samples that it's run across.

▲

manmal 4 hours ago | parent | next [-]

I don’t think this is a result of the base training data („the internet“). It’s a post training behavior, created during reinforcement learning. Codex has a totally different behavior in that regard. Codex reads per default a lot of potentially relevant files before it goes and writes files.

Maybe you remember that, without reinforcement learning, the models of 2019 just completed the sentences you gave them. There were no tool calls like reading files. Tool calling behavior is company specific and highly tuned to their harnesses. How often they call a tool, is not part of the base training data.

▲

spagettnet 4 hours ago | parent [-]

Modern LLM are certainly fine tuned on data that includes examples of tool use, mostly the tools built into their respective harnesses, but also external/mock tools so they dont overfit on only using the toolset they expect to see in their harnesses.

	▲	manmal an hour ago \| parent [-]
		IDK the current state, but I remember that, last year, the open source coding harnesses needed to provide exactly the tools that the LLM expected, or the error rate went through the roof. Some, like grok and gemini, only recently managed to make tool calls somewhat reliable.

▲

xscott 5 hours ago | parent | prev | next [-]

Of course I can't be certain, but I think the "mixture of experts" design plays into it too. Metaphorically, there's a mid-level manager who looks at your prompt and tries to decide which experts it should be sent to. If he thinks you won't notice, he saves money by sending it to the undergraduate intern.

Just a theory.

▲

victorbjorklund 5 hours ago | parent [-]

Notice that MOE isn’t different experts for different types of problems. It’s per token and not really connect to problem type.

So if you send a python code then the first one in function can be one expert, second another expert and so on.

▲

dotancohen 2 hours ago | parent [-]

Can you back this up with documentation? I don't believe that this is the case.

	▲	pixelmelt an hour ago \| parent [-]
		Check out Unsloths REAP models, you can outright delete a few of the lesser used experts without the model going braindead since they all can handle each token but some are better posed to do so.

▲

7 hours ago | parent | prev | next [-]

[deleted]

▲

r0b05 6 hours ago | parent | prev | next [-]

This is such a good explanation. Thanks

▲

hbarka 2 hours ago | parent | prev [-]

>> Same reason that "Pretend you are an MIT professor" or "You are a leading Python expert" or similar works in prompts.

This pretend-you-are-a-[persona] is cargo cult prompting at this point. The persona framing is just decoration.

A brief purpose statement describing what the skill [skill.md] does is more honest and just as effective.