While this is cool, can anything be done about the speed of inference?

At least for my use, 200K context is fine, but I’d like to see a lot faster task completion. I feel like more people would be OK with the smaller context if the agent acts quickly (vs waiting 2-3 mins per prompt).

▲

jeffhuys 3 days ago | parent | next [-]

There’s work being done in this field - I saw a demo using the same method as stable diffusion does, but then for text. Was extremely fast (3 pages of text in like a second). It’ll come.

▲

wahnfrieden 3 days ago | parent | prev [-]

Meanwhile the key is to become proficient at using worktrees to parallelize agents instead of working serially with them

	▲	i_have_an_idea 3 days ago \| parent [-]
		Sounds nice, in theory, but in practice I want to iterate on one, perhaps, two tasks at a time, and keep a good understanding of what the agent is doing, so that I can prevent it from going off the rails, making bad decisions and then building on them even further. Worktrees and parallel agents do nothing to help me with that. It's just additional cognitive load.