Remix.run Logo
noelwelsh 5 hours ago

Wow. Not a Haskell user, but a big user of other languages with expressive type systems (mostly Scala; some Rust). My experience is the complete opposite. I can't imagine using a language without a good type system to catch all the junk the LLM produces. In fact I thought people would move away from languages from poor type systems, like Python, given the cost of using languages with expressive type systems has decreased with LLMs.

em-bee 5 hours ago | parent | next [-]

exactly, i find the article a wierd take. i would have thougt that being able to catch errors at compile time is the assurance that the LLM generated code is actually decent.

so does this mean that the LLM writes code that is so good that the compiler does not find any more errors?

or is it due to the nature of haskell that makes it hard to write bad code to begin with?

or just that because the haskell compiler catches more errors there is less broken haskell code for the AI to train on?

and what does that mean for the switch to python? if the python compiler/interpreter doesn't catch as many errors do we even know that the code is good?

or is this more like the belief if the LLM can generate good haskell code, surely it can also generate good python?

what's the solution here? speeding up the haskell compiler? if that were easy, would it not already have happened?

personally i still don't trust LLM code generation. i didn't learn haskell yet, but what i hear about it makes me more likely to trust that LLMs can generate good haskell code than python.

i believe the future in LLM code generation is code that can be proven to be correct. proving code correct has been a research topic at some point.

calebkaiser 4 hours ago | parent | prev | next [-]

I think the author largely agrees with you re: type systems and LLMs. He's pretty explicit that Haskell should be very well positioned to be a power language for LLM-assisted programming, but that the Haskell ecosystem presents the bottlenecks that make it harder.

I don't personally use Haskell for anything, but I use Lean and occasionally some other languages with expressive type systems, and like you I've found it to be a pretty great experience for working with LLMs. But I've also experienced what the author is talking about, with languages that sit at different points on the type system spectrum, regarding a languages ecosystem/infra layer becoming a bottleneck. I don't think it's ultimately about the type system but the broader ergonomics of the language/ecosystem.

So I think his criticism is less than expressive type systems are a pre-LLM concept, and more that Haskell has an individually bad "agentic coding story".

roenxi 4 hours ago | parent | prev | next [-]

It is a bit surprising, I'd have guessed the same. Although in hindsight I could believe that type systems aren't particularly strong as an anti-bug layer. They help. They're a big boon for coordinating large numbers of mid- and low- skill programmers though because it forces them to go further in documenting their function signatures and makes it much more obvious where the problems are when refactoring spaghetti code because things break loudly.

Refactoring spaghetti has become easier in the LLM era because it can just read all the code, and there is now a skill floor on the programmers that kicks in somewhere relatively high. The benefits of type systems might have suffered because of that.

ffreire 5 hours ago | parent | prev | next [-]

IME Python has been very pleasant to use with types, even though they are not nearly as expressive as Haskell. I've noticed a shift in my own work where I spend more time playing with/manipulating change than I do making sure things type check. That does happen, of course, but it happens with less frequency then when I was writing Haskell by hand. During that time, I'd have stack running tests on file change and it was pretty smooth as well, but that workflow breaks down a bit with the current generation of agent harnesses we have.

koito17 5 hours ago | parent | prev | next [-]

Recently had to touch a Python project at work. Just setting up the editor needed me to use 2-3 tools out of: pyright, basedpyright, ruff, ty, mypy, and possibly other tools I'm forgetting that kind of do the same thing but throw errors in different parts of the codebase.

Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago.

In fact, one my company's greenfield projects decided to use TypeScript instead of Python for the [surprisingly] more consistent tooling, and the fact that the big LLM providers all have official TypeScript SDKs anyway. Also, for agentic coding, LLMs don't seem noticeably worse at TypeScript than Python.

My experience can be summarized as:

- for some reason we need 2-3 static analysis tools just for typechecking

- no tool understands each other's comment directives

- each tool reports a different error in your codebase

- even big libraries (e.g. matplotlib) make half their functions return Any

- you'll be tempted to silence the "partially unknown type" warnings, and you'll have to do it for each tool that's running.

mjr00 4 hours ago | parent | next [-]

> Also, for some reason Optional[T] became deprecated, just as the ecosystem finally embraced types ~3 years ago.

Optional[T] is now T | None. Means exactly the same thing but doesn't require an import. Support for the older syntax presumably won't be removed for a long while regardless.

These335 4 hours ago | parent | prev [-]

I grew up on python and after working with Java I really came to appreciate types. However I do want to point out that big libraries like mpl pre-date most efforts for typing, so it is no wonder that they arent typed properly. A lot of these libraries are trying to improve this but it will just take some time.

mpweiher 4 hours ago | parent | prev | next [-]

TFA:

The type safety we gave up hasn’t been noticeable in any concrete way yet, especially considering our test coverage has never been better.

ErroneousBosh 4 hours ago | parent | prev | next [-]

> I can't imagine using a language without a good type system to catch all the junk the LLM produces

One approach would be to not use LLMs.

giraffe_lady 5 hours ago | parent | prev | next [-]

I'm pretty sure that's the general trend and it will continue.

But I do think what benefits LLMs is the speed and accuracy of feedback. Type systems cover the accuracy part, but haskell was killing them on speed. It seems like a strange choice to go so far the other way on accuracy when there's a lot of languages in between. But I'm not familiar with the project so not in a position to call it.

It's not also really about expressiveness IMO. I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript.

Tarean 4 hours ago | parent | next [-]

Java can also have 15+ minute cold compiles on large projects if you kill all caches. It's less bad on smaller codebases because you don't have to recompile dependencies if you target a bytecode vm, but if you always gate feedback on a cold compile in a fresh VM you just aren't gonna beat an interpreted language

But I'd look at people a bit oddly if they said: 'We didn't want to set up CI caching and compiled languages took 30 minutes per run so we changed our entire codebase to python'.

Maybe it makes sense for them, and caching across dynamically spawned VM's is admittedly a harder problem which most build systems aren't great at, but still. I can easily believe that getting build caching to be reliable would be a lot of work, but is it more work than a full rewrite of a significant codebase?

pjmlp 4 hours ago | parent [-]

Additionally in modern Java there are even the options of AOT and JIT caches, which can be reused across runs.

Or if staying on Linux, JVM snapshots.

saghm 4 hours ago | parent | prev [-]

> I've found LLMs to be best with more constrained type systems: they are better at ocaml than they are at typescript.

When the potential set of behaviors you could write a program to have is infinite, but the actual behavior you want is singular, a programming language is more importantly defined by which ones it eliminates up front than which ones it lets you write (assuming it lets you write the one you want at all, but that's almost always going to be the case for most general purpose languages). Bugs are just false positives in this framing, where the program you wrote seems like the one you wanted, but there's some divergence between what you thought you were getting and what you actually got, and catching some of those up front is a huge part of why type systems are so useful.

jimbokun 4 hours ago | parent | prev [-]

You should read the article before replying.