Remix.run Logo
colechristensen 8 hours ago

Ultimately the harness is me and the experience is like managing an unruly toddler. You have to pay attention to what it does and issue corrections. Skills and prompts and AGENTS.md do some, nested sets of agents do some, but over it all is me keeping track of what it's doing and needs to do.

You have to size the unit of work to its useful attention span, you have to have code architecture that is conducive to units of work, and you have to have tools like ticket managers, etc. that keep the big picture and smaller pictures in mind. Reverting from agile practices to more pre-planning whole project documentation and architecture decisions helps.

But in the end it's you. LLMs have their limits and need humans to direct them.

maccard 7 hours ago | parent [-]

Gotcha, so no.

bmurphy1976 3 hours ago | parent [-]

You don't need a "blog post." You need experience managing projects, tracking progress, coordinating conversations, mentoring less experienced engineers, etc. That experience comes with practice and time.

If you really need something to latch onto, there's plenty of educational material available about how to be an effective scrum master. Start there.

colechristensen 2 hours ago | parent [-]

Indeed. I'm saying the human inside the engineering loop is essential. If a person needs training, particularly if the set of high level advice I set out in a reply is rejected, a blog post isn't going to cut it anyway.

Some AI skeptics paradoxically insist humans are irreplaceable and then get almost mad(?) when you discuss how humans are necessary in the loop. At least that's my interpretation of what is going on, it's hard to tell.

When building with LLMs I'm mostly product manager and QA, the whole task is having the judgement to evaluate and correct course. You can share advice but it's not a copy-paste situation where one set of prompts/harnesses/whatever solve all your problems forever. They have to be adjusted for a particular situation.