Agents are also controllable to a degree. How agents author PRs is now an engineering problem.

But this is my point, they're not really controllable in this way. There are no fixed rules (like tests or linters) for small PRs, it's not good enough to do it by line count.

A good PR tells a story to the reader that leaves them confident in the result. That means knowing the reader and how they think, and that is something LLMs are very bad at.

▲

latentsea 3 hours ago | parent | next [-]

And you can't think of a way to bring an engineers mindset to solve that?

If you leave it up to raw LLM outputs you're not going to get the results you're after. If you put on your engineering hat and think about your problem statement and what tools you can bring to bear on it you'll find you can likely craft a PR authoring skill and a set of evals for that skill that satisfies your requirements with a high enough degree of reliability.

	▲	danpalmer 2 hours ago \| parent [-]
		I'm not coming at this cold, I've worked on the skill for PR (CL) creation in the Google monorepo. I'm not talking about raw LLM outputs here. It's a hard problem that is fairly fundamental to the training material being used for LLMs today. I'm sure we can improve things, I'm sure it will get better over time, but I'd say we're a small fraction of the way towards where we need to be, and trending in the wrong direction with long trajectory development creating bigger changes on average.

▲

dhorthy 4 hours ago | parent | prev [-]

yes well said