It's not clear to me how useful of a signal replicating existing pieces of well-known software is for this kind of evaluation, given what we know about how effectively LLMs can retrieve data from their training corpus and style-transfer it across different settings (programming languages here). That would explain their convergence in ability across different languages on the tasks in this post. I'd be far more interested in people's real-world experiences.

▲

lowbloodsugar an hour ago | parent [-]

I tried writing an AI harness in Python. Seemed the obvious way to go. Tons of libraries. Libraries for talking to model APIs. Libraries for context and conversation management. Libraries for talking to MCPs. It is the language for LLMs!

It was a shit show and just couldn't write anything that would not crash. Super confident it had done a good job. Full of random bugs. A UI needs interactivity, interruption, handling exceptions. It produced some of the worst code I've ever seen. And looking at the libraries' code: also some of the worst code I've ever seen.

I switched to rust + tauri. In about three person weeks of work I have UI with forking conversations, tool use with built in grepping, tons of quality tools. It's more productive (for me) than Claude Code (CLI or desktop).

	▲	gr_norm 16 minutes ago \| parent [-]
		Yeah, I've had similar experiences, also with Rust. If the LLM will write a lot of the code for me, why not choose something (1) super fast, and (2) which has types I can use to understand and specify the code I want without having to read all the output? I've been trying out Lean for related reasons, to good effect.