Getting Completions

[00:00:00]
>> And then from there we are going to do our completions, which is very similar to the same completions we have always done. There is really nothing like new here other than the fact that we have this thing called functions. So, let's talk about that. So, const, what did I call it?

[00:00:21]
Yeah, getCompletion. getCompletion, And this is gonna take in that message that we'll use eventually, and then, what am I returning here? I was returning a response. I'll just say return openai. I'm sorry, openai.chat.completion.create like that. We do have to pick a specific model here. Not all models are equipped with function calling.

[00:00:58]
I think the one that I picked here was turbo-0613. If you go look up. There are docs on function calling, it says 4-0613 and 3.5, 0613 have function calling capabilities at the time of writing of this blog post. I think there's way more now that have that, but I'm just gonna pick one of these.

[00:01:19]
So I'll pick 3.5, 0631, I think everybody should have access to that one. Four, you might have to be on a premium plan, I'm not sure. So I'll pick that one, 3.5, 0613.
>> Isn't that just one they introduced it? Shouldn't the newer model still have it too?

[00:01:37]
>> They did, yeah, they introduced these models for the specifics of function calling, yeah, 100%. Messages, there's just going to be messages, nothing crazy there, and then functions. So this is where it's something new. So there's functions and then there's function call. We'll talk about the two. For now, we're gonna do functions.

[00:01:58]
Functions is just going to be an array of objects that represent a schema in which there is a function that GPT can call. This is just you describing the capabilities of a function. So in this case, I'm teaching GPT like, Hey, there's a function called calculate. Here's when you know you need to run it.

[00:02:21]
You need to run this function if you need to execute a math expression. Right, so let's do that. So I'll say name, I'm gonna call it calculate. I'm calling it calculate because it matches with this right here. It'll be a lot easier to check that string and this object because they're the same name but you can call whatever you want.

[00:02:41]
It's easier to match that way. Alright, we got name, we got description. Description is important. These are like this is you're gonna get this. This is the prompt that tells GPT if it needs to run this function, so like describe what this function is for. So when GPT looks in its bag of functions like, I need to do something You can like yeah this one runs expressions I need to run an expression they asked me an expression I need to run that function.

[00:03:05]
So I'm gonna say run math, expressions, right? Parameters are you describing how you want GPT. Because when you've got to respond back like here's the function I wanna run and here are the arguments you need to run it, so I have to describe what those arguments look like well my function takes an object with an expression that should be a string.

[00:03:27]
So that's what I'm gonna describe, so I'm gonna say parameters is a type object and I can say properties, Right, and then for this one is just expression like this. And then from there, that's an object with a type and a description. This is also important too. So type is string for obvious reasons, that's important.

[00:03:52]
Description, this is you get to get this is the prompt for this property, this is to tell GPT. How do you come up with this value? I think it could probably figure out because I running a math expression. This is called expression. So, and this is a string so I probably want you to give me an expression here, but you kind of wanna steer it right so here I put.

[00:04:14]
The math expression to evaluate, like, and here's an example, right? So, like, I copied that, so I could, like, there you go. The math expression to evaluate, like, 2 * 3 + 21 / 2 to the second power. I'm giving an example. I'm trying to, like, this is when you would call this, and this is what.

[00:04:36]
If you're going to call calculate, and you're gonna pass me the expression property on this object, your expression better looks something like this. This is what I expect it to look like. So I'm like guiding it right? Cool. So we got that. The next thing I wanna do is here you can say, where is it at?

[00:04:55]
I think it's here, required? No, yeah, it is required. Yeah, it's in a parameter. Inside the parameters, I can say. Required like this, and I can say the properties that are required. So in this case you when you call this function, you have to pass the expression parameter.

[00:05:21]
There's no alternative you have to never call this function without this property. Otherwise, I cannot run this function. It's not it's not optional, right? So that's what required is, and then of course we wanna put the temperature on zero because we need this thing to do serious work.

[00:05:42]
So put temperature on zero. And you can add as many functions as you want. There's also another one called function call and this is a way to force GPT, no matter what prompt they get, to call this function by name. So in this case, if I were to put this, it doesn't matter if the problem was like high, it has to call calculate and fit and then calculate has to have an expression.

[00:06:07]
So I don't know what it will put and a temperate zone zero. So I really don't know what it would put, we could try that out. But this is like forcing it. And this is actually a good hack to get structured output from GPT is actually the best way to get structured output.

[00:06:18]
So I use that to like, because if you think about it, this forces GPT to always respond back with structured output, and nothing else. So if you just need to get like some structured output from GPT, you can do this hack, and then you just never send back the results to GPT because you got back the structure output that you want it.

[00:06:34]
It's great for that. But that's, that's like a some advanced hack. So cool. And then from there, we just need to set up a loop. So we'll just say, you know, let response. And it will say, well, true. Don't blame me if your computer dies. I'm not responsible.

[00:06:58]
And then we can say response, equals await, getCompletion like this, with the messages, cool. What you talking about? Why is that?
>> The equal sign?
>> Yeah, I'm like, whoa, there we go. I'm tripping out okay, there we go. And then from here what we need to do is we can look at this property called finish reason.

[00:07:30]
I talked about this earlier. GPT is done when it's finished reason to stop just because it responded back on I mean, it's done. It might be in the middle of a function call. It might be in the middle of still generating but it stopped halfway through because I ran out of tokens.

[00:07:44]
It's only done when it says my finish reason to stop. So that's when we'll break the loop. Right so we can do that. You can say, okay, cool if response.choices0.finish_reason = stop, then we know we're done, right? And then from here, they answered the question at this point.

[00:08:07]
So let's log it. So log response.choices[0].message.content. Let's log that you. And then we'll, we'll break this loop. We're done. That's our break case, okay? Otherwise, if they respond back with a finished reason of, hey, I'm doing a function call, okay, now we need to call that function. So let's do that.

[00:08:34]
So elsif response.choices [0].message.function call, right? Or finish reason, I'm sorry, equals, or I guess it's not message, it's just finish reason. Function underscore call, like that. Okay, come here. Thank you. So, if the fittest reason is function call, as in, GPT is like, I need to actually make a name, I need to make a function call.

[00:09:18]
Based off the prompt that you gave me, I need you to call a function for me. So what we need to do is just get the name of that function, get the arguments of that function, go see if we have a function with the same name. If so, parse those arguments, call the function with those arguments, get the results of those functions, and then pass it back up to GPT.

[00:09:42]
I'll be passing that up just by pushing to this message array, which then loops again, and then it comes back up here, and it calls the completion again with new messages, and then it'll run again, right? So let's do that. So let's get the function name, and let's get the arguments.

[00:09:55]
So function name = response.choices. I should have saved this in a variable, honestly. I'm kinda regretting it now, but I'm gonna stick to the script message.function-call.name, right, so that's the name there. And then the args are gonna be pretty much that, which the arguments. So we got that.

[00:10:28]
And then we find the one that matches our function. So. FuncToCall = functions(fName) And then, arguments = JSON.parse, I guess I can't use arguments, params, I guess. So we got to parse that because it's gonna be a JSON string. And then let's actually call the function. So FuncToCall what the params boom.

[00:11:17]
And from there like I said, we got to add to the history, the results that the AI responded back with. Remember the AI responded back with a function call whose name is this whose argument is this? And then our answer to that function call, which is also real function and the name of that function, but then the string of fire result.

[00:11:39]
So let's do that. So we'll say messages.push, role, assistant, content should be null, and then function underscore call, right? And then the name is function name and then arguments are args. All right, so we added that. And then now our response, messages.push, role: 'user', name I'm sorry, role function.

[00:12:23]
Yeah, sorry. I'm like, no, that doesn't make sense. Role function name is Fname, and then the content is gonna be a stringified version of the results. So JSON.stringify result. Give me some results. Notice this result, never. Okay and again this is gonna run on a loop right? So we push to this message that comes back up here.

[00:12:54]
It runs it again with a new messages in it, and it should hit stop the second time and respond back with an answer. So let's see how this works. So first things first, we can say node functions and then I'll say what is 10 * 100 / 12 + 29 / 67 * 2.

[00:13:24]
>> It's in quotes.
>> I need quotes, I'm like, whoa, that should have worked. There we go. Let's run that and math out of that. I must have imported that wrong.
>> I think it's evaluates misspelled.
>> Yeah, evalutate.
>> You don't know how to evaluate [LAUGH]
>> I think you need to return the message in your get completion function as well for the response.

[00:13:55]
>> This? You are right there.
>> Yeah evaluate. That's That's right. Look at a word so many times, it is like doesn't look right. Okay, look, it looks like it ran. And we can test this out, right? Let's just log in here just to make sure. Hello, Yeah it ran that function, hello.

[00:14:24]
You can see the result of it is this. So now if I ask you something unrelated to this it should not run that function because it didn't need to. Spooky, yeah, it didn't run that function. It didn't need to, because I didn't ask it to like calculate something.

[00:14:47]
So I didn't feel the need to run that function. Succeed is pretty powerful. Or like, I don't know if people like understand like the crazy stuff you could do with it like you can't do anything. And this function this is an API call somewhere. This is a database call somewhere.

[00:15:01]
I don't know, this is an insane thing.