You're missing the biggest suggestion from the GP's post: that the key to LLM success is verifiability.

Software has high verifiability through compilers, linters, static analysis, leak/memory corruption detection tools, or the very least, running the program to see if it crashes.

Math (as has been making the headlines recently) has high verifiability through automated theorem provers. Notably some of the non-mechanized math results from LLMs have been demonstrated to be wrong (and presumably many more are being checked; these aren't exactly easy to understand proofs).

Game playing (chess, go, Star Craft, etc.) is similarly verifiable: you get your score and you always know how you did in the end (even if not in the moment).

Making assumptions about LLMs in non-veriable domains is making a claim about LLMs that doesn't match their current capabilities. Maybe they'll get there and maybe they won't; I can speculate as well as you can but we're both guessing, so I'm not sure how much value the speculation has.

▲

dwattttt 3 hours ago | parent [-]

This is the same highly verifiable domain that has delivered reliable, bug-free software to date?

And LLMs won't take the easy way out (like so many people) and suppress errors, change tests etc, tweak code until warnings go away?

▲

otterley 2 hours ago | parent | next [-]

A lot of testing and verification was avoided in the past because it wasn’t considered nearly as valuable as shipping code. Many of our jobs are evolving to building verifiers of AI output.

▲

dwattttt 26 minutes ago | parent [-]

> A lot of testing and verification was avoided in the past because it wasn’t considered nearly as valuable as shipping code.

AI is changing how the code is written, but I'm not seeing anything at all changing the incentives around testing and verification.

	▲	otterley 25 minutes ago \| parent [-]
		Depends on the business, I suppose. We’re all about that at mine, and I think it’s for the best. It’s definitely better for our customers.

▲

fuzzzerd 2 hours ago | parent | prev | next [-]

I routinely see AI taking the easy way out. Writing mountains of code to make tests turn green based in odd and unintuitive side effects.

That's not to say it isn't useful, and if youre on top of it you can guide it away from doing that.

▲

autoexec 3 hours ago | parent | prev [-]

If you believe the hype/marketing around how scary "AI" is LLMs are perfectly happy to go rouge and lie, cheat, and even blackmail to protect themselves.