Tool Calling

[00:00:00]
>> Josh Long: Now I'm going to back this change out because remember, this is supposed to be conversational, so you might do strongly typed things in a specific case. But for this case I'm going to make it conversational again. By the way, if you do this, remember to register at GraalVM Native Image hint for this type, right, because this needs to be reflected upon.

[00:00:20]
Okay, content, so now we've got our system, we've got the ability to find Prancer, which is great. But what's the natural next step here? The natural next step is gonna be you want to adopt this dog. Surely you wanna take this dog home as quickly as possible. So we need to give the model the ability to talk to your tools, to talk to your local APIs, to ask questions about scheduling a pickup for this dog.

[00:00:43]
Okay? So let's create a patent pending, super-unique, industry-unique dog adoption scheduling thing, okay? Scheduler. Okay. And we're going to have a method here that given the dog ID and the dog name will return a time when you can come and pick up the doggo from a particular location.

[00:01:06]
Okay. So plus three days or. No, not days. Yeah, days. Let's do that. That way it's three days from now, so we'll know whether this worked. I'm gonna say scheduling dogId and dogName, okay? I'm gonna export this as a tool. Now, when your mom always told you to use your words when you're angry, it's true for everything now, right?

[00:01:29]
If you want to do, if you want to be a good programmer, you also need to use your words, right? So here's how we do that. Schedule an appointment to pickup or adopt a dog at a Pooch Palace location. There we go. And obviously this is one of those cases where multi line strings in Java, long overdue.

[00:01:59]
I'm glad they're here, is really nice. Okay, so that's my description. And you know, I'm going to annotate these tools as well. I'll say the name of the dog, ToolParam, (description = "the id of the dog"), okay? That's terrible formatting. There you go. That's a little bit better.

[00:02:20]
Whatever. Okay, so that's my tool. I'm gonna tell my model to invoke this method to answer the question, when can I pick up this dog, whenever it needs it, right? So I'm going to provide that as a tool here. I'll just inject that in the constructor, Scheduler. And then down here, when I make this call, I'll say Tools, Scheduler, okay, restart.

[00:02:41]
And also, like I said, I'm doing some things as the default globally for all uses of this chat client, and some are per call site, and you can pick and choose. I could have specified the tools down here at the call site as well. Just your preferences of lifetime and scope and all that.

[00:02:56]
Okay. Do you have any neurotic dogs? Yes, we got Prancer. Fantastic. When can I adopt Prancer from the Minneapolis location? Am I spelling that correctly, by the way? Is there two P's or anything like that? That would have been awkward. Okay. Whoops, forgot the quote. Okay. It says I can pick up Prancer on April 26th, which is three days from now, okay, so scheduling 45 and Prancer.

[00:03:37]
So the method, the AI model was smart enough to know. Okay, I need to call this method. I need to give it the ID and the dog name, and then I need to take the response and incorporate that into my response. You can also return strongly typed objects.

[00:03:50]
I'm just returning a string to make it easy to kind of grok what's going on here, okay? Okay, so this is all well and good. This is tool calling. Super powerful. Hopefully you can see you can let your model talk to both your business logic through these tools and through your data through the vector store and stuff.

[00:04:07]
That process, by the way, that pattern of taking data that's in your database and using it to inform the response, that's called RAG, R-A-G, retrieval-augmented generation, okay? It's a very common pattern, and 90% of the use cases are just that, right? When you hear about AI engineering, it's doing stuff like that.

[00:04:25]
It's doing that and tool calling, which I just showed you.