Have you been using those models? I've been using a hand-rolled orchestrator with Mimo v2.5 (I seem to be paying $0.017 per million/tokens after their heavy caching) and it's been very impressive. I started with it in Opencode as a harness, then had it build its own micro-harness with stdlib-only Python, then used that to build a local stdlib-only Orchestrator with CLI and web harness, and now I'm using that for improving itself and now multi-project wider-ranging software. I talk to a steward who investigates and plans, then the plans are handed off to parallel worker agents who go through a work, test, interrogate, review, eval state machine for quality (all autonomously) with me at the end just reviewing the work or getting notified if the work items aren't progressing due to the workers getting stuck. So far the only "getting stuck" has been bugs/configs on my part, all at a pretty great quality bar, and at a price that makes me laugh at things like Opus.

I'm still using Claude at work (they're the only approved provider), but wow are the smaller models starting to SMOKE the big ones. At this point, all I'd consider paying out of my own pocket for is the lowest-limit Anthropic/GPT plan to get a big model as the Steward, but I wouldn't pay for ANY of the Anthropic models as the workers who do all the work. And as time passes, I don't know if I'd even do that; the open models are serving SO well.

▲

lifeisstillgood a day ago | parent [-]

So you are using a “cloud” provider and at 1c per million tokens …

Love to hear more about how you structure the orchestrator etc

▲

lelandbatey a day ago | parent [-]

Yes, it's a "cloud provider" but it's a cloud provider running an open model you can download (and that other cloud providers do host). I just happen to not have a computer big enough to host it.

As for the Orchestrator, it's pretty simple. In essence, it's like "Jira/Trello/Kanban on autopilot". Work items have states, a state machine defines how those work items transition between states, states are todo, in progress, retrying, reviewing, code reviewing, done. work items also have connections, allowing the LLMs to specify a dependency graph, and the dependency graph informs the dispatch order/parallelism, as well as when branches have to be merged. I talk to the steward, the steward has tool calls for interacting with all the data, and the orchestrator auto-dispatches all the work that comes in. I can generate work as fast as I can describe it to the steward, and that's usually the bottleneck.

So far I haven't had to deal with "how do you get the LLM to re-organize the work mid flight due to a worker finding something not accounted for by the planning", but I assume it'll come soon. The most complicated digraph I've tossed at it was 9 items and 4 layers deep. The kind of work I've given it hasn't been scoped large enough yet, so we'll see how it tackles that.

▲

lifeisstillgood a day ago | parent [-]

Ok, so I have to try that.

How are you specifiying the graphs? Is this on github (I am still trying to move from concept to how to actually do it (plus Inhave only just woken up and need coffee :-)

▲

smartbit 3 hours ago | parent [-]

Leland is a CS student, that being said I agree the concepts and pricing are tempting. There are more of these Kanban for Agents eg Multica, PlateSpinner, KitKot, Kanbots.

	▲	lelandbatey 2 hours ago \| parent [-]
		I was a student whenever I wrote my old bios but I've been a developer professionally for over 10 years at this point. My inspiration was the mayor idea from Gastown, plus wanting to formalize the informal workflow I used with agents and Jira at $dayjob.