Sort of off-topic, but is there any generative AI for code? From my limited understanding, the code is trained on human written code, and the model adapts it to what most closely matches.

What I'm curious about is, can it find innovative ways to solve problems? Like the infamous Quake 3 inverse-sqrt hack? Can it silently convert (read: optimize) a std::string to a raw char* pointer if it doesn't have any harmful side effects? (I don't mean "can you ask it to do that for you?" , I mean can it think to do that on its own?) Can it come up with trippy shit we've never even seen before to solve existing problems? That would truly impress me.

Take a bloated electron app, analyze the UI, and output the exact same thing but in C++ or Rust. Work with LLVM and find optimizations a human could never see. I remember seeing a similar concept applied to physical structures (like a small plane fuselage or a car) where the AI "learns" to make a lighter stronger design and it comes out looking so bizarre, no right angles, lots of strange rounded connections that almost like a growth of mold. Why can't AI "learn" to improve the state of the art in CS?

> Take a bloated electron app, analyze the UI, and output the exact same thing but in C++ or Rust. Work with LLVM and find optimizations a human could never see. I remember seeing a similar concept applied to physical structures (like a small plane fuselage or a car) where the AI "learns" to make a lighter stronger design and it comes out looking so bizarre, no right angles, lots of strange rounded connections that almost like a growth of mold. Why can't AI "learn" to improve the state of the art in CS?

So such things already exist, and for me, the most frustrating thing about LLMs is that they just suck the oxygen out of the room for talking about anything AI-ish that's not an LLM.

The term for what you're looking for is "superoptimization," which tries to adapt the principles of mathematical nonconvex optimization that AI pioneered to the problem of finding optimal code sequences. And superoptimization isn't new--it's at least 30 years old at this point. At this point, it's mature enough that if I were building a new compiler framework from scratch, I'd design at least the peephole optimizer based around superoptimization and formal verification.

(I kind of am putting my money where my mouth is there--I'm working on an emulator right now, and rather than typing in the semantics of every instruction, I'm generating them using related program synthesis techniques based on the observable effects on actual hardware.)