| ▲ | dhorthy 12 hours ago |
| P.S. nobody asked but since you made it this far - the next big problem in this space is fast becoming, what else do we need to be able to build these "headless" or "outer loop" AI agents? Most frameworks do a bad job of handling any tool call that would be asynchronous or long running (imagine an agent calling a tool and having to hang for hours or days while waiting for a response from a human). Rewiring existing frameworks to support this is either hard or impossible, because you have to 1. fire the async request,
2. store the current context window somewhere,
3. catch a webhook,
4. map it back to the original agent/context,
5. append the webhook response to the context window,
6. resume execution with the updated context window. I have some ideas but I'll save that one for another post :) Thanks again for reading! |
|
| ▲ | ehsanu1 11 hours ago | parent | next [-] |
| Temporal makes this easy and works great for such use cases. It's what I'm using for my own AI agents. |
| |
| ▲ | dhorthy 11 hours ago | parent [-] | | ah very cool! are there any things you wish it did or any friction points? What are the things that "just work"? | | |
| ▲ | ehsanu1 3 hours ago | parent [-] | | Essentially, you don't need to think about time and space. You just write more or less normal looking code, using the Temporal SDK. Except it actually can resume from arbitrarily long pauses, waiting as long as it needs to for some signal, without any special effort beyond using the SDK. You also automatically get great observability into all running workflows, seeing inputs and outputs at each step, etc. The cost of this is that you have to be careful in creating new versions of the workflow that are backwards compatible, and it's hard to understand backcompat requirements and easy to mess up. And, there's also additional infra you need, to run the Temporal server. Temporal Cloud isn't cheap at scale but does reduce that burden. | | |
| ▲ | dhorthy an hour ago | parent [-] | | helpful - thanks! I have played with temporal a bit but have this thought that since most AI tools represent state as just a rolling context window, maybe you don't have to serialize and entire call stack and you can cut a bunch of corners. but we're all probably better off not investing that wheel |
|
|
|
|
| ▲ | lunarcave 8 hours ago | parent | prev | next [-] |
| The MCP[1] that was announced by Anthropic has a solution to this problem, and it's pretty good at handling this use case. I've also been working on a solution to this problem via long-polling tools. [1] https://github.com/modelcontextprotocol |
| |
| ▲ | dhorthy 8 hours ago | parent [-] | | thanks for bringing this up. I just spent 2 hours last night digging into MCP - I'd love to learn more about how you think this solves the HitL problem. From my perspective MCP is more of a protocol for tool calling over the stdio wire, and the only situation it provides HitL is when human is sitting in the desktop app observing the agent synchronously? Again, genuinely looking to learn - where does MCP fit in for async/headless/ambient agents, beyond a solid protocol for remote tool calls? | | |
| ▲ | potatoman22 5 hours ago | parent [-] | | You could implement some blocking HitL service/tool as an MCP server. | | |
| ▲ | dhorthy 4 hours ago | parent [-] | | ah okay - I guess in that case, I would like chain a HitL step as an MCP server that wraps/chains to another tool that depends on approval? or is there a cleaner way to do that? | | |
| ▲ | a1j9o94 2 hours ago | parent [-] | | Yeah, exactly. You would define a HitL server and the actions it implements would be API calls to your system. | | |
| ▲ | dhorthy an hour ago | parent [-] | | this is interesting. I will have to think more about how humanlayer can support an MCP integration/wrapper, it's not immediately obvious to me i do think that AI-calling-tools is insufficient to provide bidirectional communication rails for user input/review though...not disagreeing just maybe thinking out loud a little here |
|
|
|
|
|
|
| ▲ | soheil 2 hours ago | parent | prev [-] |
| Just to frame the problem slightly differently, if you had unlimited number of humans who could perform tasks as quickly as a computer this wouldn't be a problem need solving. So since we know that's the end state for any human-in-the-loop system then maybe it's worth solving that problem instead. A few things come to mind, divide the problem into chunks that can be solved in parallel by many people. Crowd source your platform so there are always people available with a very high SLA, just like cloud servers are today. |
