Remix.run Logo
▲ mmastrac 3 hours ago

Yes, absolutely. The problem is integrating it into the inference engine. Not impossible, just _very hard_.

▲Majromax 3 hours ago | parent | next [-]

Also less useful than one might think at first.

Lots of code "typos" aren't immediately syntactic errors. Misbalanced parentheses, for example, only bite when one needs to close the series. The solution is to go back and fix the original error, but that option is foreclosed to autoregressive LLMs. In the meantime, forcing the model off what it "wants" to generate might make the remaining syntactically valid tokens semantically meaningless, impairing future generation in a way that's hard to quntify.

Constrained generation for tool calls is certainly a type of syntax checking, but it's one that can be justified because the "hard" syntax choices come up front, both in the very format of the tool call itself to the name of the tool and its parameters. Errors are more likely to be detected in the same root-cause tokens that generate them.

▲ 3 hours ago | parent | prev [-]
[deleted]