> <200ms for 45 questions at once

Considering your own question length: ~120 characters x 45 divided by 4.1 ~= 1317 tokens.

So question processing at 5.5k PP(around the actual PP speed of GPT5.6 Sol) it would take around ~0.24 seconds + the context processing.

Computing the output should be around ~20ms (at 50 tok/s), computing 45 tokens in parallel.

> have 0% malformed output

Pretty trivial; only the allowed output is selectable :)

So, I keep repeating myself: Jev was a low-hanging fruit all along; no one cared, and probably no one will in a few weeks?

Yeah but a lot of developers who didn't even know that this was a possibility now do, and will probably find use cases for it.

You can probably even share context between questions by cleverly manipulating the attention mask.