Create an Agent Tool Call

[00:00:00]
>> Scott Moss:So we talked a little bit about the anatomy. Again, this is just what it looks like. It's a description. There's a schema, and then there's an execute function. And now we're going to get into just making the tools ourselves. So, one thing I remember in the previous lesson, I was going through the .env file. You might have seen some other API key in here. Don't worry about this, we'll get to it. You do not need it right now.

[00:00:23]
It's not in here by accident. But yeah, we will get to this and if you're reading this and you're copying, don't worry, I'm going to change this after the course, you're not going to take my API key. But yeah, we'll get to this later, so don't worry about it. But anyway, let's head over to our run file, which is where our agent is going to be, and we want to add in some tools here. But first, let's make a tool.

[00:00:45]
So to do that, you can go to your tools folder, you can click on it, there's an index here, it's an empty tools object, that's fine. Let's make a new tool. It's like the Hello World of tools, which is basically like a date time tool. Most agents or LLMs don't know what the current time is or date unless the provider themselves are injecting that into the inference call when you hit it on their server, and maybe they do, maybe they don't.

[00:01:09]
So it's best sometimes just to have a tool that an agent can lean on to get the current date time. You can also kind of stick this into a system prompt, but this is just a hello world tool. So let's just go ahead and do that. So the first thing we want to do is import this tool helper function from AI. This makes it super easy for us to create a tool, and then we're going to import something called Zod.

[00:01:30]
If you've never used Zod, it's just a runtime schema maker that we can use to validate objects. That's it. Or validate any primitive, any value. This is a schema. It's like a database schema but at runtime, right? It's like the de facto one in the JavaScript land. And next, we're just going to export a tool called dateTime that's going to use this tool helper like so, and we're going to write those three things that we said.

[00:01:54]
The first is a description. Now, this description matters because this is essentially a prompt to the LLM. The same way you would have typed a prompt to the LLM is the same thing that you would type into the tool description. You're describing what this tool does and when an agent might use it. So, let's think of a description that would be helpful to let the agent know when it could use this and what it might do.

[00:02:29]
So I'll say, returns the current time and date, right? And you could also add some more stuff here. Use this before, use this tool before any time related task, right? Like I want to make sure you use this tool. Like I'm trying to codify a workflow with English, right? I'm trying to tell you, hey, make sure you use this, but it's always just a suggestion. It's never a rule, right? This isn't deterministic.

[00:02:55]
There are no wrong answers here. When we get to evals later on how to improve this, you'll understand why, how much influence this has, but for now, put whatever you want. We'll make an input schema. Input schema, at least for this library, pretty much always has to be an object, even if it doesn't have any arguments. In the case of getting a date time, we don't need any arguments. You could maybe have something called like, you know, time zone and it takes in a string and then you can describe to the agent, you know, the time zone you want, right?

[00:03:30]
You could do that and then it would pass that in, but for this example, we don't need that. And then an execute function, which is the actual function that's going to get executed when the agent asks us to run this tool, and we'll keep it simple. We'll just return a new date here, like so. And we'll just say toISOString. Now, the reason we return the string here is because the output, the input to an LLM is always going to be text.

[00:04:01]
You can only place text into a prompt. One, because, you know, that's the only, that's serializable over the network, but two, it's because LLMs understand language. That's the second L in LLM, large language model. They don't understand anything that's not language. So giving it language is what we want. So we want to try to best figure out what is the right output of a tool to help the agent understand this.

[00:04:34]
Like right now I just put this, you know, as a string, but if I wanted to be more expressive, I could say like something like, the current date time in ISO format is, and then I could do that, right? To be more expressive. But there's no right or wrong answer. Once you get to evals, you'll, we'll learn how to improve this and measure it to see what the differences are. But you can put whatever you want.

[00:05:00]
I'll just have it what I had it in the notes just to be consistent. Cool, so once we have that, we want to go ahead and register that here in our toolbox. We'll just say dateTime. Make sure you put a .ts on that import. We are using modules, ES modules in Node. And then once we have that, we just need to create a function that will, given a tool name and an argument, we'll do a switch statement on all the given tools that we currently have, pick the right one and execute it.

[00:05:34]
That way we don't have to do that in our agent logic. So inside of our tools folder, we'll just create a new file here called, actually, just on the root here of the agent. We'll say executeTool like this, and then like I said, it's mostly just going to be something that's going to do a switch statement on all of our tools. So let's import all of our tools from the index. Here, cool, we got that. And then we're just going to export const executeTool async function.

[00:06:15]
It's going to take in two arguments, the name of the tool and then any possible arguments that the LLM is passing in for that tool, if that tool has any arguments. We're going to get the tool from tools like this and I'll just call this toolName. I have a type for that. Actually, I'll just make one. Well, you know, I'll just say any, we'll just any of that. So we have our tool here. If this is not a registered tool, for some reason you're passing in a tool that we do not have registered, then what we can do is, for now, we can throw an error, but instead of throwing an error and breaking your application, we can tell the LLM something to hint at it that like, hey, this is not a known tool.

[00:07:11]
So I can say unknown tool, you know, use something else or like, you know, unknown tool, this does not exist. Whatever you want to hint at your LLM, you know, you can do that. After that, we want to get the execute function, so we'll say execute, and that will be like tool.execute. Like so. And then not every tool will have an execute function, as you'll learn, as we go through the course, so we probably want to check for that.

[00:07:48]
So if not an execute function, then this was just like a tool that is mostly for stopping a process or anything. It doesn't actually do something. So in that case, maybe we also want to hint to the LLM that like, oh, you know, this is not a registered tool, there's no wrong answer here, it's language, you can put whatever you want. And we'll fix this with evals once we measure it. And then the next thing is, if there is one, we want to get the results.

[00:08:28]
We can call that by awaiting the execute of that tool, and then we can pass in the args if there are any. And then optionally we can pass in a tool call ID. Don't worry about this. We'll talk about this later. It's just the SDK wants us to pass this in to satisfy it as context. And then messages. Cool. And then I'm going to make this any. Or not as any, I'll just say any here. Nope, not any there.

[00:09:18]
Let me fix this type because it's not going to compile if I don't fix this type. Export. So I'm just going to fix this type here. We're going to say toolName equals key of, what is it, keyof, oh yeah, type of tools like so. There we go, and then we can say as toolName. And then we can fix this type. We just need to return the results and just to be a stickler, we'll just wrap this in a string just to make sure it's a string.

[00:09:47]
That's probably not going to work all the time because maybe it's a JSON and you need the JSON stringify. But honestly, it should be the responsibility of the tool itself to make sure that it returns a string and not the responsibility of the execute tool function to make sure a string is returned. Just make sure you return strings in your tools. Cool. So now that we have that, we can go over to the runner.

[00:10:21]
And we can continue to use generateText, but now what we're going to do is we're going to import in our tools from the tools index, like so, and then we're going to import executeTool from the executeTools. And we're going to pass in a new thing here called tools. And it's just going to be tools. There are some other things here that we can do as far as like tool choice. And there's a couple of options.

[00:10:46]
By default, it's going to be auto. This means let the LLM decide which one of the tools it should pick. None means don't pick any tools, and required means you have to pick at least one tool, right? These are all different options. By default, it's auto. There's also something called approved tools, or where is it at? Or I'm sorry, active tools. This is where you can put an array of registered tool names of like, hey, I know I gave you a bunch of tools, but only pick from these active ones.

[00:11:18]
So there's a lot of ways to configure it. By default, it's auto, it's going to pick whatever one it wants. And then from here we'll get something called tool calls. So once we get these tool calls here, we will no longer get a text. So if the LLM decides that it wants to call a tool, it won't generate a text, it'll generate a tool call, which will be an array of objects of tools that it wants to call.

[00:11:46]
So let's log both. I'll log tool calls and I'll log text, and I'll pass in the tools, and then I'm going to ask it something that should trigger it to use the date time tool. So what is the current time right now, right? So I'll do that. I'll go back in here. I'll run this. And you can see we did not get back a text, we got back an array of objects. This is a tool call. It's an array because by default, the LLM can run parallel tools.

[00:12:12]
It can decide that I want you to run these three tools in parallel at the same time. We can turn that off if we want, but we'll just leave it on. And it's basically saying it's like, hey, I have a tool called, here's the ID for it. The ID is important. It gets generated on the server from the provider. We have to use that ID to identify the results for this tool when we add it back to the LLM. So you have to keep track of that ID.

[00:12:38]
The tool name is the one that we gave it. It's called dateTime. So it's saying, I want to call the date, I need the date time tool to answer this question. And here's the input. The input is a blank object because we said the date time tool has no input, so it doesn't have one. And this other stuff is just metadata, right? So now that we have that tool, what you could do is you could just call executeTool, right?

[00:13:11]
So in this case, you could say toolCalls, you know, forEach. This is just a very oversimplification of what a loop would look like. And then for each one of those, you could just call executeTool, pass in the toolCall, toolName, right? There aren't any args, but I'll pass in the input anyway here, right? We'll do that and then actually I'll just log it so we can see it. Like that, so we can execute that.

[00:13:47]
And then I'll run this again. And if we did it right, we'll see that, I need to call await, obviously. Inside a for loop, I know, anti-pattern, it's fine. We can see that that tool ran and we got the current date time in an ISO string, right? So this is just a single turn. What we've done so far is a single turn. Remember in the beginning when I showed you the agent where it just kept thinking and then picking a tool, getting the results and then thinking again, that's a multi turn.

[00:14:19]
It's doing this on repeat. So far this is a single turn because cool, the agent, the LLM said, hey, I need to run this tool, and we're like, cool, got it, I ran it. But how does the LLM know about the results of that tool? We have to feed it back to the LLM to be like, hey, we ran that tool for you, you remember that tool that you wanted for this ID? Here, here are the results for it. Is there anything else you want me to do?

[00:14:44]
That's what needs to happen next, and we need that to be on the loop indefinitely until it either reaches a conclusion, errors out, hits a max turn, time out, whatever the condition is. We need to feed it back, right? And that's the next thing that we're going to do. But first, before we get into that loop, we want to make sure what we currently have on a single turn actually works. And when I say works, that it has some degree of quality.

[00:00:00]
So before we get into the loop, what we're going to do instead is write evaluations and learn about evaluations for a single turn LLM process.