Approval Flow Architectures

[00:00:00]
>> Scott Moss:OK, two different types of flows. We have synchronous and we have asynchronous. We're going to do synchronous. It is not the best flow. It's terrible. It doesn't give us any of the benefits that I just talked about, but we're going to do it because we're in a terminal. But basically the agent loop will suspend and pause, but the process is still running. It's waiting, right? And, you know, again, you're going to be bound by your transport and the system that you're running on.

[00:00:22]
In the case of a terminal that's waiting, that thing will sit there and wait as long as we don't interrupt that terminal process. If we shut our computer, if we quit that terminal process or whatever happens to our computer, it's done. It's not durable, it's out of there, it's synchronous. Nothing else can happen while it's waiting on your approval. It can't move forward. The loop is literally suspended, right?

[00:00:44]
That's a synchronous flow, it's super easy to implement. It does not scale to multiple users who want to pause their entire server while someone needs to respond to something. It doesn't work at HTTP because you'll have a timeout. And the user must be there. This is for like this only works like for instance, this would work in a self-driving car because you're in the car. So when the thing pops up it's going to be like, OK, yes, right, because you're there, so I don't feel bad that I'm doing a synchronous approval because you're in the car, I guarantee that.

[00:01:16]
But again, that's not true freedom. I mean, in a car you should be there, I guess, but like if you want to do something else, not as dangerous as driving a car, but just like, hey, can you just send this damn email for me? I don't want to be sitting there watching you interact with my Gmail. Like I could have just did that. What was the point? And I think that's a big argument for a lot of people is like, sure, this thing can do that, but I still had to watch it and approve and like do all, I could have just did that.

[00:01:37]
I really wasn't that helpful. That's a synchronous flow. It's really great for CLI tools, local development, and like real-time interactive sessions where I'm definitely here, self-driving cars, you know, something where you know I'm for sure here. Asynchronous, the future. Or background agents, these are things that are stateful and persistent and have durable executions. So when they ask for an approval, they just stop.

[00:02:04]
They just shut down, there's no compute, everything's saved in persistent storage and you can approve it at your own leisure, whenever you want, a year later, it doesn't matter. And when you approve it, it spins back up, it sees that you approved it and it continues where it left off. It might even be like, man, so much time has passed, I need to like look at my world again and see what has changed because I'm already in.

[00:02:23]
I actually just a year ago and so much has probably already changed and I'm already on step 20 so I probably need to like redo this, right? That is possible today. It's just people aren't building them, unfortunately. That is the future. This is what allows you to be like fire and forget. I can send off some tasks, I can ask an agent 20 different things, and I can go on with my day and then it'll just update me throughout the day just like one of my co-workers would with like a question or an approval or some more context or an alert that it's done or they're like, hey, I wasn't able to complete this, and then you just get to it when you want to get to it because you're not wasting money, the compute has shut down, it's not a person on the other line that's really needing you to do this, and they're just there for you, so if you don't want to respond to it, then you just won't get it done at that point.

[00:03:13]
If it was urgent to you, you would respond, it's that simple. I think that is the future of what agent looks like, and there's like a hybrid approach where like, there's some quick decisions that can happen like, hey, I need you to do this right now versus something in the background, right? So, it gets very convoluted about like on what surface you want to do it, right? So you can do it like on a per tool approval, so like every call to shell command, you have to get an approval.

[00:03:44]
This is why in our tools index file I was exporting them in different groups, so you can like, oh if it's a file group tool, approvals. If it's a terminal group tool, approvals, right? So you can approve on different groups or categories or single commands. You can do input-based approvals where it's like, OK, I'll auto approve a shell command if it's just these things. If the thing that you pass into the shell command is just ls or cat auto-approve, you don't need to ask me for that, right?

[00:04:15]
If the thing that you pass me to the read file command does not include this secret path that I don't want you to see, yeah, auto approve, you can read that. I don't care, right? So you can base it off inputs as well. You could do session-based auto approved. This is something like Claude Code would do where it's just like, yeah, trust this tool for the rest of the session or trust all tools that match this pattern or, you know, user grants, escalation permissions, you know, as trust builds.

[00:04:43]
You have like risk scoring, that gets complicated, and then, you know, time and cost base. This is like enterprise level stuff where it's like, OK, auto approve this if the operation takes less than this amount of time. That means you need some way of computing how long something might take. You might just hard code that on a per tool basis, you might estimate that based off of, we've seen these in the logs over time and the average time that these things take to run is this, so we'll put that in there and then the AI can determine whether or not that is true or not.

[00:05:12]
We know how much these APIs cost, so we can see that. So there's a lot of different ways you can make that dynamic. And here's the cool part, it's the notification channels. If you do the background agent stuff. You can notify someone anywhere. You can send them a message, an iMessage, you can call them, you can send them an email, you can slack them, you can make a custom client which they get inputs.

[00:05:34]
There's really not much you can't do because everything is distributed and disconnected. You can't do that with a synchronous flow. And I go on to talk about the future of agent inboxes, you can read that, but that's essentially the startup that I was working on.