Remix.run Logo
▲ micbas 9 hours ago

Naive question: why do we care about programming languages when we no longer read programming languages directly? Shouldn't we either design languages optimized for LLMs? (needs fewer tokens, easier for bots to process, easier to machine verify, not sure what else)

▲profunctor 9 hours ago | parent | next [-]

We do still read code. Most people are not doing a fully unseen development strategy because we have needs around efficiency, security, compliance etc and we can’t simply blame it on our agents like we don’t control them like OpenAI do

▲pjmlp 9 hours ago | parent [-]

That is like arguing everyone should master Assembly when using high level languages. :)

▲wakamoleguy 9 hours ago | parent | next [-]

Which tangentially brings up another analogy: many high level languages don’t compile directly to assembly but to an intermediate language first (LLVM). We don’t read the LLVM code ourselves.

▲rlander 7 hours ago | parent [-]

But LLVM compilation is deterministic. LLM code gen isn’t.

▲pjmlp 7 hours ago | parent [-]

Only when not used alongside PGO data or as a JIT, as its inputs are not deterministic in first place.

▲profunctor 8 hours ago | parent | prev | next [-]

No it isn’t. I think you read into my comment too much.

▲creata 6 minutes ago | parent | prev | next [-]

I don't want to say what people should and shouldn't do, and a world in which all the world's professionals spend their days talking to sycophantic machines with only a loose commitment to the truth is really funny to me.

But, high level languages don't really lose precision in any way that matters (except when you need extreme performance, resistance to timing attacks, or niche stuff like that). Natural language on the other hand loses precision that's usually important, so I don't see how your comparison is valid.

▲spider-mario 9 hours ago | parent | prev [-]

Who said “everyone”?

▲pjmlp 8 hours ago | parent [-]

We do still read code. => We do still read Assembly code in the world of compilers.

▲spider-mario 8 hours ago | parent | next [-]

Not seeing it. “We do still read code” implies neither that every single person does, nor that they need to master the language being read. Taken together, it seems to me that you need a fair bit of extrapolation (or a presupposition that programming languages are already obsolete when that’s the very point being contested) to make the analogy work.

▲Topfi 8 hours ago | parent | prev [-]

Deterministic vs None-Deterministic, understanding what compiled code does also is possible without Assembly knowledge, not so necessarily with models, where I do like to at least understand what has happened and quite often find a crime or five.

Heck, OpenAI can't even get their models to reliably do what they intend, so yeah.

▲pjmlp 8 hours ago | parent [-]

AOT without PGO, manual memory management, traditional optimisations vs AOT with PGO, JIT, tracing GC, machine learning optimisation passes.

▲Topfi 8 hours ago | parent [-]

Which have what to do with the relationship between assembly and the higher level language?

▲pjmlp 8 hours ago | parent [-]

It is the answer to you Deterministic vs None-Deterministic.

If you don't get the difference, I get there isn't much that you know about how instruction selection works, across all those approaches.

▲Topfi 8 hours ago | parent [-]

Ok, I feel you know what this is actually about, but here spelled out:

Do any of the optimizations you mentioned affect the underlying behavior of the run code? Semantics...

If not, it doesn't fit to LLM use, where this is very much the case. We can talk about bugs in compilers and undefined behaviours and so on and so forth all the livelong day, but you know as well as I that this is not of the same scope of impact.

I could and in the past would have written a long diatribe about how pure compilation to assembly is deterministic, thus making this not really suitable and that while yes, there are things like JVM that aren't 100%, we have contracts, docs, specs and, because of them, an expectation of consistency that LLMs do not deliver. I did not because I thought that was not necessary, but see my error now, that is my mistake.

▲pjmlp 7 hours ago | parent [-]

There is the observable behaviour, and the machine code instructions that actually get generated in every single case for the same kind of text input.

Which for example, unless you can exactly replicate the world every single time a PGO data gathering session is done, there are no guarantees that the compiler will generate exactly the same machine code pattern, when given various PGO files generated at different occurrences.

▲Topfi 7 hours ago | parent [-]

Thanks, yes. To get back to the "We do still read Assembly code in the world of compilers" analogy, in the case of LLM usage, the observable behaviour is...

▲pjmlp 7 hours ago | parent [-]

Assembly naturally,

https://www.phoronix.com/news/Frame-X11-Server-Assembly

▲Gualdrapo 9 hours ago | parent | prev | next [-]

Relevant Calvin & Hobbes strip:

https://static0.cbrimages.com/wordpress/wp-content/uploads/2...

▲logophobia 9 hours ago | parent | prev | next [-]

I mean, even if you've stopped even reading code (good luck with that), the right abstraction should make things things easier for the LLM. A good rich type system to stop incorrect code, linters to catch common errors. A good set of battery-included system libraries. BEAM should provide a good abstraction for highly parallel and distributed systems, which means the LLM doesn't need to build all those abstractions itself. If you're building a distributed system, it should be like providing a set of good guardrails, guiding the LLM in building a correct system.

I've seen even the best models try to build & debug parallel and distributed systems. The results are always a bit uneven at best, even with a good harness and a lot of guidance.

The disadvantage of niche languages is that the LLM won't quite know the language as well due to less training data being available. But if you actually need to build something distributed, then this might be a good trade-off.

▲wakamoleguy 9 hours ago | parent | prev | next [-]

We might still read them, but let’s set that aside. Some languages may offer ways to eliminate entire classes of bugs. Gleam and Haskell-ish languages do that through very strong type systems and expressive high-level functional constructs. Rust similarly eliminates a ton of memory errors through its borrow checker. LLMs aren’t infallible, so this can result in better quality code (or if you believe every bug will still be caught…less code to do it).

Gleam also runs on BEAM, which has important runtime characteristics vs other languages. That will impact your system and how it scales even if you never look at the code.

▲WJW 9 hours ago | parent | prev | next [-]

Who's this "we" you speak about. There is a small subset of programmers who have become effectively become managers and ask their AI employees to do things. The rest of us still regularly read and write code.

▲jeremyjh 3 hours ago | parent [-]

I’d love to see some numbers on this, but this is not my impression from the communities I participate in (such as the Elixir forum.). Amongst programmers who are employed, I suspect it is a majority at this point who use agents to write code. It is hopefully a small minority who never read code, but it’s certainly growing.

▲nchagnet 9 hours ago | parent | prev | next [-]

Even in an AI intensive workflow, reading code remains a huge quality gate and I'd be wary of any software which hasn't been reviewed by humans.

Moreover, to me coding carefully is still a hobby, a pleasure, a craft. Gleam is exactly the kind of language that makes this such a pleasant activity.

▲benrutter 9 hours ago | parent | prev | next [-]

I'd make two cases:

1. In a lot of cases, even if you're pure vibecoding and not reviewing any code, its still useful to be able to read it - you can then verify and reason about various aspects. LLMs are often flat out wrong on niche issues of performance etc, so for cases like this, not being able to read code is a big risk.

2. "Languages optimized for LLMs" sounds awesome, but so far, to me at least, it seems like LLMs work best in the same situations as humans- good test coverage, low-complexity, clean architecture, etc.

▲OtomotO 9 hours ago | parent | prev | next [-]

> why do we care about programming languages when we no longer read programming languages directly?

Because, we do?

You may not, which is fine. A lot of developers may not... that's fine. I guess you're all employed and don't risk your livelihood when things go wrong.

I do though. I take responsibility.

And just like with any junior developer I guide AI and I do review what it creates in the project that matter. (Personal tools I've entirely vibe-coded, never any client projects though)

▲Topfi 9 hours ago | parent | prev [-]

For what it's worth, I still parse code changes and feel uncomfortable if I struggle to understand code output without assistance.

On the "languages for LLMs front", isn't that a bit "cart-before-horse"? Doubt we are anywhere near the limit of what a model could do with a static verifiable language, even more so since stuff like TS has massive training data, any new language wouldn't benefit in such a way.

Add to that, I'd rather be able to understand the output, partly for aforementioned reasons, partly because we have seen with post Spud GPT reasoning/summarisation issues what happens when being able to read model output becomes even slightly harder. Having model-focused languages would only exacerbate such issues.

Also, absolutely not a Maths expert so take this for an utter layman regurgitating of what I have read, but apparently the Lean proofs by OpenAI are very hard to parse even for the experts in the field, lovingly called "Clanker Proofs", even when they are valid.

Having not just correctness but also human understanding as a goal has merit, so there is that to consider too. Rather models get more efficient at using what we have (and can understand+maintain as humans if need be) then make it more opaque.