Yeah all of these attempts to try and control AI outputs by using language fail to grasp that language is itself the problem. It is a closed system with no ground truth. Words only point to other words in an endless self-referential chain. There is no "closure" in language, ambiguity cannot be erased.

Imo LLM researchers would benefit a lot by reading what continental philosophers have said on the topic of language. Barthes, Sarte, Deluze, Derrida, et. Al have a lot to say that explains why we're running into the same problems over and over.

Natural language is ambiguous. Specifying exactly what you want in a way that is truly verifiable is the act of coding.

Bend would make Dijkstra happy even when proof checking can’t verify if the laws are what was actually meant.

I actually think Asimov is more instructive here, while Gödel and Tarski tell us the tool can’t prove itself…

Nonetheless, it is a worthwhile endeavor and I hope more rigorous practices like this catch on.

I think LLM researchers understand how LLMs work and what the limits of using natural language as an interface are. The problem is everyone else thinks they're basically magic, expecting them to be infinitely intuitive but also strictly deterministic, like the computer from Star Trek.

[dead]