Remix.run Logo
▲ dachris 19 minutes ago

And (from a training material point of view) it makes perfect sense.

I would assume that to a LLM, a binary executable is not much worse than actual source code. And it's much denser, just missing the semantics of human-written identifiers. It's the code's pure logic, condensed into machine language.

And there's tons of it, many vendors supply it "freely" (you don't need license keys to train on a binary) It's often battle-tested code that you wouldn't get hold of otherwise, and compared to ingested source code its unlikely that Codex regurgitates a recognizable part of a Windows 2000 DLL into your C++ source code.