Remix.run Logo
Der_Einzige 3 days ago

Correct - Dynamic grammar based/constrained sampling can be used to, at each time-step, force the model to only make valid moves (and you don't have to do it in the prompt like this article does!!!)

I have NO idea why no one seems to do this. It's a similar issue with LLM-as-judge evaluations. Often they are begging to be combined with grammar based/constrained/structured sampling. So much good stuff in LLM land isn't used for no good reason! There are several libraries for implementing this easily, outlines, guidance, lm-format-enforcer, and likely many more. You can even do it now with OpenAI!

Oobabooga text gen webUI literally has chess as one of it's candidate examples of grammar based sampling!!!