Simple AI Chat in Node.js

[00:00:00]
>> All right, so, if you go to the Simple chat UX section on here, I have most of the code here, if not all of it. I think there's one part right here that I guess I was going to put the code in here, but then I decided not to.

[00:00:11]
I don't know, but we're still gonna do it. So you can follow along, but the best thing is just to follow along with me. This is just here to reference if you wanna catch up, also the repo is an exact replica of everything that's in these notes. So you can look at that as well.

[00:00:25]
But probably just follow along with me because I'm probably gonna reference this. So I write it the exact way that I did so there's no confusion, but I might get off track. I get off track a lot. So we will see. So let's do it. First thing I'm gonna do is, I'm just gonna make another file.

[00:00:41]
I'm just gonna call it chat.js. And I'm also gonna make another file. I'm just gonna call it openai.js. And I'm just going to move our openai configuration in this file so we don't have to make it every single time. I don't wanna make a new OpenAPI instance every, we're gonna make a lot of examples.

[00:00:59]
I just don't want to make one every single time. So I'm just gonna make it once and then we can export it. So I'm just gonna import dotenv here, config, and then I'm going to import OpenAI from openai, and I'm just gonna export const openai = new OpenAI like this.

[00:01:20]
There we go. So that way that's just there and we don't have to make it every time. Okay, so inside of my chat, I'm just gonna import that openai from the file that I just created. I also think because we're using type modules you have to do the .js extension, otherwise it won't work.

[00:01:43]
Pretty sure if you tried to import this file without .js it'll break if type is module. Try and let me know. But I'm pretty sure at least on this version of node, maybe they fixed it on other versions. On my version it will break if you don't do .js.

[00:02:00]
We also need to, wait, no, it's built in now. I was gonna say we need to install readline, but it's actually built into node now. So we can just import readline like this. So we say, import readline from node:readline. Okay, if you're looking like, what the hell is this, if you've never seen that before, this is optional.

[00:02:23]
This is something new in Node. This is basically saying, hey, I know that Node has some internal package called readline. That's the one that I wanna import. And the reason this is great is because there might be a package you installed called a readline. In fact, there was one, they had to rename it because Node made one.

[00:02:43]
So if there's an internal thing called readline and there's also a package you have installed called readline, which one is it gonna import? I don't know. So, yeah, to get around that confusion you do that. So now it will always import the one that's internal to node and not some third party one that you installed.

[00:03:00]
So you can actually put this node prefix on every internal module that node has to force it to use that one versus trying to figure out whether it's gotta import the one that's installed with the same name or the internal one. Because some crazy person decided to name their NPM package the same name as an internal node package.

[00:03:20]
Could you imagine, an NPM package called FS? Who would you do that? That's crazy, but people do it. So, all right, we got that. readline is just a way that allows us to write text in our terminal and see text output, that's basically it. We're gonna use that.

[00:03:40]
We already initialized this, we're good to go. So now we just need to make a new readline interface. So the way you can do that is I'm just gonna say readline = create. Oops, readline.createInterface like this. And you can say input and for the input, basically, we're gonna use process.stdin and then stdout.

[00:04:05]
This is just basically how a terminal handles input and handles output. It's like a pipe through which it can receive text and in which it can output text. We're basically binding that to the inputs and outputs here. So input is stdin. Why did it bring that in like that?

[00:04:28]
I didn't do process, that's right. Process .., and then output, if I can type, is the process.stdout like that. So cool, now we have our interface. We can start making some chat stuff. So from there, what we wanna do is, I'm just gonna create a helper function that wraps the call a call to the OpenAI API, to make a new message.

[00:05:06]
That way, we don't have to write this every single time we want to make a new message. I'm just gonna create a wrapper around that because that's annoying to write all the time. So newMessage, it takes a message, right? And then from here, we're just gonna say, response or results, whatever you want to call it.

[00:05:28]
This is gonna be async = await openai.chat.completions.create, right? Also, because we need to pass in the entire history for everything, we're gonna take the history as well. So when we create this, we'll say model, I'm still gonna use gpt-3. You can literally use whatever you want. And then I'm going to pass in the messages here.

[00:06:05]
The messages are gonna be a brand new array of everything that came before, so the history. Plus your new message that you passed in like this. New messages go on the end of the array, not the beginning. You would not get the results you wanted if you did that.

[00:06:26]
All right, follow me so far? Okay, and then yeah, we just wanna go ahead and just return the message object itself. So that's what I'll do. I'll say return results.choices.[0].message. So basically the same thing we did in the index file, nothing different here at this point. But the next thing we need to do is we need to create like a loop that runs constantly so we can keep having the conversation.

[00:07:03]
Whereas before on our previous example, we'd ask a question and it will stop, the process will stop. But we want it to keep going so we can keep the memory alive and we can keep having a conversation. So that's basically what we need to do next. But before we do that, I'm just gonna create a function that formats the user input that someone types in.

[00:07:26]
So we'll say const formatMessage. It takes in userInput, which is just a string, right? And then it's just gonna return a new object here that has role of user, and then content of the user input, Like that. So if you're looking at like what is going on here, if this looks new to you.

[00:07:57]
This is just me doing an implicit return. If your arrow function only has one line of code, you don't have to put the return statement and then because I wanna return an object, I can wrap it in a parenthesis. So therefore I return an object. If I didn't have those parentheses, it's thinking I'm making a function body and I get a syntax error.

[00:08:17]
But if I put it in parentheses, I'm saying return this object. So it's just a little shortcut. Cool, everybody follow me there? All right, so the next thing is we just wanna create the main chat function, which is going to be the thing that houses our loop. And this is where we try to keep things going.

[00:08:40]
So the way that that's gonna work if you scroll down, this is where we're going to keep our history. And then we're gonna have this start function here that is recursive. And this is how we recreate our loop. We're just gonna use recursion as a mechanism for keeping things going.

[00:08:58]
And that's how it's gonna work. So if you've never done recursion before, if you've done it, wow, when am I ever gonna use it? You're gonna use it right now. This is that one time ever in your life that you're gonna use recursion and actually is beneficial and hopefully won't crash your computer.

[00:09:12]
Well, I guess recursion never really crashes your computer because it reaches a call stack limit whereas a while loop would. Cool, okay, so let's do that. So const chat and then underneath it I'm just gonna call it chat eventually like that. And let's create our history, which is just an empty array.

[00:09:33]
But for now, let's prime the system with a system message. So I'll say, role: 'system. Remember, this is the origin story of the AI. So we always got to start off with that, the origin story. So content role and content, and then put whatever you want here. I'm just gonna keep what I always put, you are an AI assistant.

[00:09:58]
Answer questions or else. I don't know, whatever you want to put there. [LAUGH] You put whatever you want. That's the beauty of it. I wonder [LAUGH] what that's. I don't know what's gonna happen there, but I kind of threatened it a little bit, so we'll see. I'm gonna get moderated.

[00:10:17]
All right, so that's the backstory. Let me put a y there. And make another function here called start, Inside of chat, and then underneath that I'll just kick it off by calling start, that's the loop. All right, and then in here, what we wanna do specifically is, we wanna go ahead and do this, our readline.question.

[00:10:38]
What this is gonna do is it's gonna prompt a question in the terminal in which you can type something in. So let's do that. So I'll say rl.question. I'll prefix the question with you. So when we see it in a terminal, we can see that this is our message, and then we'll see the log for the AI's message.

[00:10:57]
So I'll say You, takes a call back, that's gonna be async, and you get back the user input here like that. And then from this, we want a way to exit out of this when we're done. So I'm just making a handler for that. So if you type in the word exit, it'll close.

[00:11:21]
That's basically it. So what I'll say is if userInput.toLowercase === 'exit' or whatever word you wanna use to close out of this. Then you can say rl.close and then this is returned out of this. So this will end the session effectively, just by typing the word exit. And then from here, what we need to do is we need to format that user input because user input is just a string.

[00:11:54]
And we know that in order to send a message it needs to be an object that has a role in a content property. And we have that function called formatMessage that does just that. It takes a user input and it converts it to the appropriate thing. So let's do that.

[00:12:07]
So I'll say message = formatMessage takes in a userInput, right, we got that. And then from there, what I wanna do is I want to call the OpenAI API with the current history, so everything that's been said thus far, plus this new message. All right, then I wanna update my history with the new message that I sent, plus the response that I got back.

[00:12:41]
There's like a million ways to do this, so I'm just doing it this way. So I'll say response = await. What did I call it? newMessage, Await newMessage. It takes in the current history we have here. It takes in the newMessage we want to send like that, so we get our response.

[00:13:03]
And then we want to update our history, so I'll say history.push. I wanna push in first the message that we created from the user input. That needs to happen first. And then I want to push in the response that we got back from openai. So I'm updating our history with that.

[00:13:20]
So that way when this happens again, when we pass in this history, it's been updated with these messages, which works because history is outside of the start function. It's in a closure so it never gets erased. Cuz we're doing this inner function here, so we got a closure there.

[00:13:38]
And then from there, obviously we want to log what the AI is saying. That's the whole point so let's do that. Console.log. I'm gonna add in some new lines here so we can better see it. If you didn't know back tick in means new line. Backtick in JavaScript means escape this character, which means don't literally show this character, it represents something else.

[00:14:04]
So then I'll say AI, and then here we can put the result. I'm actually just gonna use backticks here. And the response here should just be response.content, like that. Okay, and then we need to run this again. So we got to call start again right underneath it, so that's the recursive loop.

[00:14:26]
We gotta run this again because now we want this to run again so we're getting asked another question. If we don't do this again, we won't get prompted to ask another question. So you need to do that. And then obviously the initial kickoff is down here. There's many different ways to do recursion, I know the official way of this pattern, but it's just, I call it the inner recursive function.

[00:14:48]
I mean, we could have made chat recursive too, but then we'd have this issue with the history of closure. We had to put it up here somewhere and then probably make it an object. It gets weird. So there we go. Okay, cool. And then lastly, you can just put a log at the bottom before you call chat just so we know that, hey, it's ready to go and you can hit exit when you're done.

[00:15:19]
Otherwise, you're just looking at a blank terminal and you won't know if it's working or not. Okay, let's try to run it. Maybe it works the first time. Maybe it doesn't. So I'll say, node chat.js. You can see right here, it says Chatbot initialized, type exit to end the chat.

[00:15:39]
That's a good sign. And then now you can see it's prompting me to ask something. Let's test this out. Hi, my name is Scott. I hit Enter and it says, hello, Scott. It's nice to meet you. How can I assist you today? You can see now it's prompting me again.

[00:15:57]
Now, just to test it, what is my name? Your name is Scott. Okay, it remembers. We're good now. So now we have an AI chat interface that remembers things, whereas before, that was not the case. So, this is a quick little ChatGPT in your terminal if you wanted to do something.

[00:16:21]
I actually made something like this similarly. And I use it over ChatGPT because I just prefer using the terminal than I did going to the web outlet. This is just better for me, in my opinion. So, cool, and then if you wanna exit, you can just type exit and then you're out.

[00:16:37]
So yeah, obviously ChatGPT will be a little more nuanced as far as its features and stuff, but the main part of being able to chat to it, I mean, this is essentially it, right? And I think I'm gonna add some new lines here. So it's right, not next to it.

[00:16:50]
Yeah, that's basically and then eventually like I said you will hit this point, depending on what model you choose of, it just won't remember anymore and you'll have to handle that. So that's a constraint, and then we'll talk about some more of the constraints in the next point.

[00:17:08]
But I just want to make sure I handled any questions here about this interface. I think the first time I did this I thought, wow, this is not that hard, this makes sense. Because doing something like this before, GPT came along, forget about it. This was not gonna happen.

[00:17:26]
Can you imagine that company Intercom having something like this when they first came out? It would be insane. It would be unheard of. But it was just really hard to do that back then, if not impossible. And now anyone with an API key can do this. And I don't know, 47 lines of code.

[00:17:45]
It is kind of nuts and you don't need to know anything about linear algebra or matrix math, what you don't want to know about. So I thought that was pretty intense.