In the last couple of days I wanted to try out the new definitive DeepSeek v4 releases. I gave it the repository of a semi-abandoned video compression codec and I told it to perform the usual benchmark -> profile -> verify -> research -> improve loop. I specifically chose this codec because the authors include a verifier for the bitstream to make sure you don't break stuff if you want to try your own implementation. I gave the agents access to the compiler's profiler and also Intel's VTune, which has fantastic output. In a couple of hours the LLM generated SSE and AVX implementations of the compression and decompression algorithms that almost doubled performance with a single core. Then I asked it to create a CUDA implementation using NVIDIA's NSIGHT profiler as a guide and it also started doing some good work.

Personally, I believe that LLMs should be treated like an advanced version of Prolog or linear programming: you give the constraints, you have a way of verifying correctness, and you give it a clear goal. If the LLM can verify itself and course-correct you can basically leave it on autopilot

I have used Opus 5 and some Fable 5 to finally get realtime transcoding of 4K 10-bit HEVC (to 1080p or smaller SDR AVC) working on a Raspberry Pi 4. It was very good at writing optimized NEON kernels. the Argon HEVC hardware decoder outputs SAND30 which is a tiled format that is annoying to work with and not really supported by anything else, the big performance issue has been with converting and scaling it, but as it turned out a lot of it was really with just moving memory around, so by fusing multiple steps into a single kernel it became fast enough. Experimenting with writing the NEON kernels for the different combinations would have taken forever by hand.

I do wonder if auto-research would have reached something similar, it did take a significant amount of steering from me to get it to the point where it was working realtime.

For anyone interested the ffmpeg is at https://github.com/poizan42/jellyfin-rpi-ffmpeg and a shim for using it with stock jellyfin at https://github.com/poizan42/jellyfin-rpi-ffmpeg-shim

I did something similar recently with Google's C# protobuf library. I had spotted I was getting CPU bound rather than memory bandwidth bound when doing streaming of uint32 buffers in dotnet gRPC.

I then asked claude to compare the C#/.NET implementation in the library with the C++ version, and it quickly identified that the C# library was missing a couple of fairly cheap optimisations that were present in the C++ version.

If I can help get a PR merged, then it'll be by far the biggest impact of any work I've ever done.

I also compared the Rust version, it had this specific optimisation. The far more popular Tokio/Prost library did not.

Given appropriate guardrails, LLMs are impossibly fast at iterating to find root causes and specific performance bottlenecks.

I tried kernel autoreasearch using DeepSeek-V4-Flash as well. It spent about 1-2 hours to complete the FlashAttention optimization job (https://github.com/fengwang/FA5090/tree/main/v7) and cost me only $0.2. I believe we are ready to offload a lot of this kind well-defined constrained optimization problems to AI Agent autoresearch.

I've had a lot of success decompiling old video game ROMs in exactly this way. Like you say - give it a way of verifying correctness - put it in a loop - and they are quite surprising.

https://github.com/qarl/arcade-js

Had a similar experience with my Rust implementation for JSONLogic expression evaluation engine. As it has a full test suite with 1000s of cases and a benchmarking script, I was able to give some basic hints to try different optimization techniques and the end result was impressive. Reached from 1.6s to 200ms for a full benchmarking test. https://github.com/GoPlasmatic/datalogic-rs First 3 versions were hand written and maintained for 3yrs, and now 4th version came out in less than a month's time with impressive performance.

This is exactly how I use it. I mean not on abandoned repos, but in a benchmark - profile - verify - research - improve loop.

I'm hoping to release a native Wine for MacOS w/ Win32 FEX support in the next few days.

Watching claude and codex play winquake and age of empires, and debug support for Firefox 52 has been wild.

Is Mojo programming language useful in the age of AI?

This is the way. Checkout the technique mentioned in the alphaevolve paper and see if it works well for your setting.