+1, llama.cpp has a --grammar parameter which you can pass a BNF style grammar file to constrain generation. It can be used in Python llama.cpp wrapper

https://til.simonwillison.net/llms/llama-cpp-python-grammars

Yes. But even then, the probabilities are not calibrated. In jev/laya, they are (well, relatively anyways).