Remix.run Logo
nurumaik 7 hours ago

Minified json would use even less tokens

vitaelabitur 6 hours ago | parent [-]

Yeah, but I tried switching to minified JSON on a semantic labelling task and saw a ~5% accuracy drop.

I suspect this happened because most of the pre-training corpus was pretty-printed JSON, and the LLM was forced to derail from likely path and also lost all "visual cues" of nesting depth.

This might happen here too, but maybe to a lesser extent. Anyways, I'll stop building castles in the air now and try it sometime.

memoriuaysj 5 hours ago | parent [-]

if you really care about structured output switch to XML. much better results, which is why all AI providers tend to use pseudo-xml in their system prompts and tool definitions