| ▲ | lukev 4 hours ago | |||||||||||||||||||
As has been said, actual evals are needed here. Anecdotally, the worst and most common failure mode of an agent is when an agent starts spinning its wheels and unproductively trying to fix some error and failing, iterating wildly, eventually landing on a bullshit (if any) “solution”. In my experience, in Typescript, these “spin out” situations are almost always type-related and often involve a lot of really horrible “any” casts. | ||||||||||||||||||||
| ▲ | resonious 3 hours ago | parent | next [-] | |||||||||||||||||||
Right, I've noticed agents are very trigger happy with 'any'. I have had a good time with Rust. It's not nearly as easy to skirt the type system in Rust, and I suspect the culture is also more disciplined when it comes to 'unwrap' and proper error management. I find I don't have to explicitly say "stop using unwrap" nearly as often as I have to say "stop using any". | ||||||||||||||||||||
| ||||||||||||||||||||
| ▲ | energy123 3 hours ago | parent | prev | next [-] | |||||||||||||||||||
The question can be asked two ways: (1) Are current LLMs better at vibe coding typed languages, under some assumptions about user workflow? (2) Are LLMs as a technology more suited to typed languages in principle, and should RL pipelines gravitate that way? | ||||||||||||||||||||
| ▲ | mewpmewp2 4 hours ago | parent | prev [-] | |||||||||||||||||||
This is why I have very specific ruleset and linting for my LLMs, not allowing any at all and other quality checks. | ||||||||||||||||||||
| ||||||||||||||||||||