I have not generated haskell code yet and did some haskell a while ago (before LLMs) and liked it a lot but I would imagine you are prompting the LLM like an engineer so to say and making sure you know the data types and everything before hand and probably tending to write type signatures and asking the LLM to complete it. I would imagine if you vibe code and ask it to write things without thinking that sort of thing though it might produce very bad code, but not sure, would like to know you experience on that matter.

▲

solomonb 13 hours ago | parent [-]

I don't know how to answer this. I find that the quality of claude's output in any language is highly dependent on your skill at prompting it and what context you can provide. Sometimes I might give it type signatures and heavily micromanage it but other times not. I almost always ask claude to develop a plan and architecture which I review before having it produce code.

Here is an example where I literally just told claude I wanted generic instances for these type classes and we had a brief back and forth about the approach then it one shotted this entire module: https://github.com/solomon-b/monoidal-functors/blob/main/src...

I would say this code is indistinguishable from generics code written by a human. However, this type of code has a tendency to only be writable one way.

▲

kodoman 9 hours ago | parent [-]

> Here is an example where I literally just told claude I wanted generic instances for these type classes and we had a brief back and forth about the approach then it one shotted this entire module

Yes this is basically what I thought you where doing (what seems to me the correct thing). I have found it's very good at implementing interfaces or in your case type classes and completing generic functions. Though I am not sure if you wrote the type class definitions in this file or not, but sounds like either way you explained what you wanted. Also this sort of Haskell code is well above what I did in Haskell so I can only take your word for it that this is good code and I am sure it is.

Though I always use plans like you, and found them to always be good. I have experimented with just asking it to implement features with minimal direct and it does end up being very bad. Particularly in creating data structures/ data types or interfaces and classes.

My guess was only that when you try and get the agent to decide stuff like data types and type classes and how to structure the program it does a poor job and worse then if you did it in other languages.

▲

solomonb 3 hours ago | parent | next [-]

I did not write the type class declarations in this file but they are generic representation for type classes I did write.

I picked an extreme example. Sadly I don't have a great example of more "business" style code in a public repo to show you right now, but I have had a lot of success using LLMs to write CRUDy haskell APIs and such as well.

I think LLM output quality is wildly dependent on how we prompt them. And I don't just mean giving a really specific design for it to implement or whatever. I have no real proof but suspect that every little thing about how we communicate to an LLM will impact the quality of the output. Like even just our choice of language in how we describe a problem.

I have a friend who is not remotely a programmer who tried to vibe code a big python project with Claude. He came up with really detailed designs and worked through them over and over with Claude. He hit some major roadblocks and asked me to look over the code. I was really shocked at the quality of the code out.

There was tons and tons of dead code, unnecessary repeated expensive computations, no real sense of an architecture, etc. I didn't even re-write it by hand, I just asked claude things like "decompose all the functions to a reasonable size", "use modules to organize the code", "remove dead code", "write pure functions", "parse dont validate", etc. It eliminated thousands of lines of crap, became way easier to read, and improved the speed of the program by orders of magnitude.

▲

nyssos 9 hours ago | parent | prev [-]

> My guess was only that when you try and get the agent to decide stuff like data types and type classes and how to structure the program it does a poor job and worse then if you did it in other languages.

It's about the same as for other languages, in my experience. LLMs are terrible at Haskell type design, but they're terrible at Rust type design too, terrible at deciding when and how to use Python classes, terrible at deciding when metaprogramming is appropriate, terrible at deciding what is well factored and what is not - great scripters, terrible lead engineers. The root causes seem to be things like bad domain modeling, judgement about which things matter and which things don't, prioritization, and so on, all of which produce bad designs in any language that will let you. The only really Haskell-specific defect I've noticed is an aversion to polymorphism and especially to HKTs.

	▲	kodoman 8 hours ago \| parent \| next [-]
		Yes agree it's not a good architect (and thankful as it puts a ceiling of crappy vibe coded apps and what vibe coders can do). But in languages like go-lang and python it's easier for you to just hand off something you want made to an llm and come back and possibly maintain and fix it in the future or modify it. I have found go-lang fantastic for this, and although I would not do this for an important thing, for writing little tools that are a little beyond what would usually be a script and you want to maintain, it's great. I did this for a tool to insert data from stripe into a db for my service and it was a one shot thing.
	▲	solomonb 3 hours ago \| parent \| prev [-]
		> LLMs are terrible at Haskell type design, but they're terrible at Rust type design too, terrible at deciding when and how to use Python classes, terrible at deciding when metaprogramming is appropriate, terrible at deciding what is well factored and what is not - great scripters, terrible lead engineers. I find there are shibboleths you can say which get the LLM into a space where it can do better but generally I agree with you. I also find that if the LLM is working in an existing code base with a really opinionated and consistently applied design that it will largely stay consistent with the codebase. Quality will drift as the ratio of human to LLM code changes so you periodically have to do a big cleanup to keep it from going too far.