Remix.run Logo
riwsky 3 days ago

> For those who’ve built coding agents: do you think LLMs are better suited for generating structured config vs. raw code?

Raw code. Use case was configuring a mapping of health data JSON from heterogeneous sources to a standard (also JSON) format. Initial prototype was a YAML DSL, based on the same theory as yours. LLMs had difficulty using the DSL’s semantics correctly, or even getting its syntax (not YAML-level syntax, but the schema: nesting levels for different constructs, and so on). It’s possible that better error loops or something would have cracked it, but a second prototype generating jq worked so much better out of the box that we basically never looked back.