I think the real disaster is that once you let the LLM work on a project for a bit, you start to lose understanding of what exactly is even happening under the hood in the project. You can take steps to mitigate this, but agents don't exactly encourage the behavior required to maintain a good understanding of what's going on.

▲

westurner 44 minutes ago | parent [-]

Like becoming a manager?

When a person becomes a manager, they do or do not have enough time and expertise to review all of the code that they trust the team to produce.

Managers usually get into automated testing; unit tests, integration tests, acceptance tests, and maybe also BDD syntax

Managers and developers are responsible for setting a test coverage threshold for merge approval.

If there is 100% branch coverage test coverage for a codebase, what would coverage-guided fuzzing or property testing find? If there is 100% branch coverage test coverage for a codebase, what is the value of spending resources on formal verification?

How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage?

	▲	tablarasa 14 minutes ago \| parent [-]
		> How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage? This is such a salient question. Sometimes (definitely not always) the test suites produced by LLMs are so trivial it's scary. Coverage can be an illusion for sure.