OpenAI Hello World

[00:00:00]
>> Scott Moss: Let's do a Hello World. I'm gonna check out to step one. You should follow along with me here. And first thing you want to do is make sure you make a .env file, add your OpenAI key in there, just literally, just like that. I mean, don't put the brackets around it, obviously but OpenAI equals and put your key if you don't know what a .env file is, this is a file where you can put your secret variables in your repo and make sure they don't get push up the GitHub so people don't take them and then get your whole system compromised.

[00:00:34]
So if you put in the .env, I've already have it ignored in the Git ignore, so you're good, but without this, you won't be able to, participate today. So make sure you have it. It has to be just like that OPENAI_API_key. Put that in the .env file in the root of your repo.

[00:00:49]
I didn't include it because the .env file's not checked into GitHub. I can include it. That's the whole point of it. So add your .env file and do that. Once you have that, we're going to just make a super simple LLM call and we're going to see how AI works.

[00:01:07]
I guess I'll just give you a quick tour of what you should see in this repo. So we have an index TS file that literally does nothing but processes a user input. If you don't give it an input, it'll freak out. I have some types in here that I use for TypeScript.

[00:01:22]
If you don't know TypeScript, that's fine. You don't really need to know. For the TypeScript folks, this is some types that I made so our editor doesn't scream at us. And, we have the source folder. Most of these files are empty. I do have a AI folder here that just instantiates OpenAI and just exports it so we have to do it every single time we wanna use it.

[00:01:42]
So nothing crazy here. Everything else is mostly empty. There's a UI file here. So we're not building anything frontend-related. As you saw in the demo, it's just all through the terminal. To make that look pretty, I made a file to do that. So obviously we aren't gonna walk through how to do that.

[00:02:00]
That's just done for you. So everything else you will have to make but this is our UI. This is the thing that I showed you in the terminal, because I don't think learning about AI has anything to do with, whether or not you can tease out some TypeScript react, or something like that.

[00:02:15]
It's irrelevant. We just wanna talk about AI. So we're gonna do that. Also be sure to make sure you install your dependencies, NPM install. I have dependencies. They're in the package JSON. There's a lot of stuff here. Well, not really, but do NPM install to install these dependencies?

[00:02:29]
Our app depends on them to work. You will not get anything to work if you don't do this. So do that. Make sure you install your dependencies. Like I said, I'm going to use Node and NPM going forward, if you want to use BUN, you can use and just point BUN to the index file and run that.

[00:02:46]
If you're using MPM, I have a command for you. It's NPM start. We can use that. So that's what we are gonna to do but let's keep moving. So the first thing we're gonna do is we're going to add a LLM function here called runLLM. And essentially what it's gonna do, it's gonna take a message and it's just going to call this endpoint here here, openai.chat.completions.create and get back the message.

[00:03:07]
So let's do that. What you can do is you can go to the LLM file here, and you can import OpenAI from AI, and cursor is looking at my Git history so it knows exactly what I want to do. And this is some of the stuff that's kind of annoying to me with these AI tools is like, how do you know I want to do that exactly?

[00:03:29]
Because you saw it in my Git, that doesn't mean I want to do that right now. And sometimes it does get in the way. But when it's right, it's cool but no, we're just going to export a function here, we're gonna call it runLLM. Like this, it's gonna be a async function.

[00:03:44]
And then basically what we want this to take is really there's only one thing that we need here it's what is the message the user is sending? Because the reason it's called a completion, if you look at the OpenAI, it's called completion, because you're completing this request, you're completing this sentence like, what is the next thing after this?

[00:04:08]
So it's called a completion. So I'm gonna take the user message here like this. See and now it's trying to suggest so much I don't even know where my code is okay there we go. And then I'm just gonna add some types here as well for my own sanity.

[00:04:25]
So I'm just gonna say that's a string. And then what we wanna do here is we want to call the OpenAI completion. So what I can do is I can say I wanna get a response here. That's gonna be awaits. I'm gonna say await openai.chat.completion.create like this. There's a couple parameters that we can take here.

[00:04:52]
I'll walk through some of them. So the most important one probably is the model. You have to go look on the open AI documentation to see what models they have. I think if you even just, Let me see if this is type checked. If you just put a string here, yeah, if you put a string here, you can kind of see all the models that they currently have.

[00:05:08]
The way that it works with OpenAI is that the bigger the number, the newer the model essentially. But just because it's newer, that doesn't mean it's better. I think 3.5 to 4o. That rule stands. So all the way up to 4o. Anything before that 4o is going to be better.

[00:05:27]
But then once you get to 4o there's just different variations. You have 4o mini, which is a small mini. I talked about models being trained in different parameter sizes. Okay, it's like the same training technique they use for 4o, which is some breakthrough that they had. And then they gave it less parameters.

[00:05:42]
So it's not as good as 4o, but it's probably better than or maybe as good as 4, all right? But it's smaller and it's cheaper. The smaller the model, the cheaper it is too and the faster that it is as well. So you got 4o mini, you got all these different variations.

[00:05:58]
So they have dates on them because when models get trained, that's the training cutoff date. So they update those models with new data. So they put the date on the model name so you know when that model got trained. So if you wanna ask that model some information up until that date, if it got trained on that data, it should.

[00:06:16]
So you just know the training cutoff date. So you'll see that. And then you have their o1, mini o1 preview. This is like the cream of the crop. Best stuff out there right now. If you ran this, you might pay a couple of dollars today. So, don't run o1 mini or o1 preview.

[00:06:32]
Those models are mostly only good for writing code. I would never use that for anything else other than writing code or doing heavy math stuff, PhD level stuff. So everything else, you can just do 4o mini for the most part, and then 4o if you want something bigger, we're just gonna use 4o mini here.

[00:06:51]
If you got big bucks, you can just do 4o, if you're feeling it, I'm just gonna do 4o mini, because it's super fast and it's really good. And then the next thing is this concept called temperature. This is just a measurement of how creative you want the model to be.

[00:07:07]
The higher the number, the more creative I think it maxes out at two. So I recommend just putting it on point one. Just don't do anything other than point one. You do anything other than that, you're asking for it. So just put it on point one. So experiment with it later, see how it goes, but just put it on point one.

[00:07:26]
It reduces the chaos. Chaos is like entropy. It's the randomness that the AI has. This reduces the randomness. It's never gonna be deterministic, but this reduces the randomness, but sometimes you do want that up, maybe when you're generating an image or something like that. Who knows. And then messages.

[00:07:45]
Messages is interesting. And we're going to dive into this a little more. But messages are an array of objects that represent the messages in this chat so far. All you need to know for now is that they're objects. They all have something called a role. In this case the role as you can see, there's a few of them.

[00:08:00]
There's assistant function, system, tool, user, because this is a user message, the role is user. And then they have a content, which has got to be the user message like that. And then finally, we just want to return response choices. It's pretty much always going to be the first thing in there.

[00:08:19]
We get back the message and we want to get the content on there. So this is what the AI responds back with. This is all you need to hit an AI. That's it. This is it. Done. You have AI in your app now. You're done. [LAUGH] Everything else is just like manipulating it and building something really cool.

[00:08:38]
But this is it. This is AI right here. Yeah, then the next thing is let's just add this to our index file so we could actually use it. So I'm going to go back to my index file. I'm going to import that in here like that. I'm going to say const response equals await run the LLM with the user message like that.

[00:09:00]
Then I'm just going to log the response. Let's see if I broke anything. So NPM start, and then say something. I'm going to say, hi. Then yeah, it says, hello. How can I help you today? Right, and then I'm going to say, what? I'm going to say, my name is Scott.

[00:09:19]
And it's gonna say, cool, nice to meet you, Scott. And then I'm going to ask it, what is my name? What do you think it's gonna say?
>> Speaker 2: Nice to meet you?
>> Scott Moss: You think he's gonna say, your name is Scott, right? Okay, sure. No it doesn't [LAUGH] because it's not an agent.

[00:09:42]
It doesn't have memory. It's transactional. We didn't feed it the previous messages. It only knows the message we sent it and that's it. So problem number one, AI is very forgetful. It has no memory by default. You have to supply that, models don't remember. So that's part of what building an agent is, is adding memory to it.

[00:10:03]
And then it's just a spider web from there. Okay, cool. Before we take a break, let me address the LangChain question that we had. So first of all, what is LangChain? LangChain is a framework that you can use to build AI applications, whether it's agents. These retrieval mechanisms, input sources, everything you need to build a production ready AI integration in your product.

[00:10:31]
LangChain strives to be that. Originally created in Python, there is a TypeScript implementation. In my opinion, I think LangChain's value today is not that high. I think it was really good if you were a Python developer back in the 3.5 days with OpenAI because OpenAI's API was still being developed and they were still trying to figure it out and there wasn't really cool things right now.

[00:10:58]
Like tool calling and structured outputs, and providing OpenAI a schema. You can do a schema now, as you'll see, and today we'll use that. They didn't have that back then, so LangChain filled that void. If you want to get something back in a very specific JSON format, LangChain had a way to do that.

[00:11:15]
It would basically just call your thing or repeat and repeat until it did it or have another LLM check it for you, but you don't have to do those things anymore. So I feel like the value is not there. I don't know, I'll probably use LangChain for like a month a couple years ago, and I've never used it again.

[00:11:30]
And the other thing is if you're a JavaScript developer, I have no idea who's doing that stuff on LangChain. But the way that it's created and the way that you use it, the APIs for it are not what you would do in JavaScript. It's very weird. It's like they ran the Python code through an AI.

[00:11:46]
It's like, convert this to TypeScript, and that was the output of it. Because it's just it's way too crazy. They have a million ways to do one thing, and none of it makes any sense. So I personally recommend not using it. And everyone that I've ever talked to that has an LLM Integration and production just uses the native APIs.

[00:12:03]
They almost never use a framework. They just make their own thing. At my startup, we just made our own thing. We just generated our own agent. And what I'm teaching today is an abstraction of what I built at my company. So it's stuff that I actually use in production.

[00:12:18]
So, I mean, give it a try. I think Personally, using LangChain is going to confuse the hell out of you, and you're probably better off just doing this stuff, and I say from scratch. But in reality, OpenAI is really good, and every other AI provider uses the OpenAI API spec.

[00:12:36]
So to replace OpenAI with Claude or something like that. It's as simple as just changing the name of a model or URL, and you don't have to change any of your code. It's exactly the same. Assuming that model has the same capabilities as the one that you're using, it's exactly the same.

[00:12:50]
So that's my opinion. I don't recommend LangChain.