Introduction

[00:00:00]
>> Scott Moss: All right, welcome everybody to agents in production. I am your instructor, Scott moss, and in this course, we're gonna be teaching you how to take a prototype agent that you built with an LLM all the way to production. So very excited about this course. This is what's been on top of mind for myself for the last year plus, every morning I wake up, I've learned some new way of incorporating an LLM in production, and it blows my mind.

[00:00:27]
So I'm gonna be sharing some of those things with you today. First, let's just kinda hop into what you're gonna need, and then I'll talk about some of the other stuff. So obviously you're gonna need get Node js latest version, 20 works. That's why I tested on, I'm not sure if anything 18 might work.

[00:00:42]
I haven't tested it, but 20 for sure definitely works. You can use bond if you want. That also works great. You're gonna need to open AI account and note that GPUs aren't free. So you're gonna have to add a credit card to that account and you're gonna have to add five bucks as a minimum to use the account, although you probably won't be going through those, yeah, you probably won't be going through those $5 today.

[00:01:05]
It depends on how many times you run these commands, you should probably be fine. But, yeah, that should be more than enough for this course. And then there's gonna be another account that we're gonna set up together. It's from a product called upstash. You do not need a credit card for that.

[00:01:20]
But I'll walk through setting that up, and we'll talk about it later. So if you're taking this course and you're like, what's an agent, never used LLMs before? Kind of heard about them, no worries. I have another course for you, and that's gonna be building agents from scratch.

[00:01:34]
So you can check out that course. And in fact, this course builds on top of that curriculum. So if you just took that course, you're in great hands. If you haven't learned anything about agents, please take that course first, you're gonna learn so much. And again, the materials is building off on that.

[00:01:47]
So check out that course. First thing is the repo. You can go here and check out the repo on my GitHub. Should say agents in production. The notes are in the read me and all the others materials that you need so you could check that out, and then prerequisites as far as like, what you need to know, again, agents, if you wanna know more about that, take the other course I just mentioned.

[00:02:10]
Obviously you need to know JavaScript. So, if you don't know JavaScript, this might be a little hard, but you definitely don't have to be like an expert at JavaScript to take advantage of this course when not doing anything, this isn't like going into detail about JavaScript. So if you just have used JavaScript or any language, any high level language similar to that, you'll probably be fine.

[00:02:29]
Although I will be using TypeScript. So if you've never used TypeScript, no worries. Types are optional. You don't have to do any of that stuff just as long as you know JavaScript. APIs, if you know how to consume APIs and use APIs are pretty much gonna be good here.

[00:02:42]
We're not gonna be building APIs, so no worries if you haven't built any. That's totally fine. It's mostly just consuming. And then again, LLMs. So hopefully you have some basic understanding of how to use LLMs. Having built an agent or something similar with an LLM will be helpful.

[00:02:55]
If you have no idea what an agent is or an LLM is, please, again, check out the course building agents from scratch that I taught here as well. But other than that, you should be good. As far as the format, like always, I'll introduce some concepts, or mini concepts, and a lecture format, all the notes, and all the details will be here in that section.

[00:03:16]
I'll kinda just give you my TLDR of it because I don't like reading bullet points, I kind of like to tell you how I feel about it. And then from there, you guys can follow along as I implement the things that I just taught you into the application that we're building.

[00:03:31]
And then as far as like, how you might follow along, you can literally follow along as I'm doing it. You can copy and paste the code that's gonna be here in the notes. So every step that we do, the code is gonna be in here. And then the last thing is, once I'm done doing the live coding, I'll commit that to push up to a branch where you can just check out if you just wanna get caught up on exactly what I just did.

[00:03:52]
So three ways to follow along. You can kinda pick your own speed there. So what are we gonna be building? I'm gonna go to my terminal. So again, it's heavily based on the other course, but it's an agent that has access to Reddit, random dad jokes, and what else was, yeah, to be able to generate images.

[00:04:13]
So those three combinations, you can imagine the results were kinda kooky. So we're building off on that agent, adding some more functionality, but more importantly, getting it ready for production. So here's the final product of that. So I'm just gonnago into some of the things that we're gonna be covering in this course.

[00:04:31]
If you wanna know more about like how that agent works and how it was built, it's in the previous course. So one of the things that we're gonna be doing is something called evals. So an eval, we'll dive into it, but essentially I'm able to, let me see, run, I could do something like this.

[00:04:50]
And what this is gonna do is, basically, you could think of as like a test. It's gonna test whether or not the LLM was supposed to do it was supposed to do. Do so without diving too much into it, because we're gonna do it, this eval just checks to make sure that the agent picked the right tool given a certain user message.

[00:05:10]
So in this example I'm saying, tell me something cool from Reddit. So I'm writing it in eval to check to see if the agent decided to pick the reddit tool versus any other tool. So I wanna be able to check that. So we're gonna be doing evals. We're gonna be writing evals for all different types of stuff.

[00:05:26]
This is literally someone's job to do this. So we'll be doing that. At the same time, you'll be able to view those evals. I built a whole react app where you can view those evals on a graph and you can see kinda how they differ, stuff like that.

[00:05:39]
So we'll be taking a look at that. I'll show you that when we get to evals,. So, that's one thing. The other thing that we're gonna be adding is rag, so retrieval augmented generation. We'll talk more about that, but basically we're gonna teach our AI, we're gonna give it access to index of data that it has not been trained on previously and then we're gonna teach it how to query that data to answer questions.

[00:06:03]
So we're gonna be indexing this agent on a movie data set from IMDB, so we can ask it questions about movies and things like that. So if I say, find me a scary movie about ghosts. If this doesn't break, it will think, it will do a movie search, and then it will try to synthesize all that data and give me a list of movies, I'm guessing.

[00:06:33]
There we go. So you can see it spit out a list of movies. So we'll be building that as well. That's something that we'll be doing. And then one of the other things that we'll be doing that I can show, because a lot of the other stuff we're doing, like you can't actually see because it's like framework stuff in the background, is we're gonna be implementing something called human in a loop.

[00:06:50]
So human in a loop is, when I say that in AI, that means so many things. But in this example, it means like an approval. So before the AI can do some action, it must seek my approval first, and I have to say yes, or some form of yes, before it can proceed.

[00:07:09]
So we wanna implement that. So usually you do that on like some type of destructive, mutative operation. In our example, our AI can generate images. So before I have it generate images, I am laughing because I'm thinking about the last course. I wanna get an approval first. So I'll ask the agent, generates me an image of a fluffy cat.

[00:07:35]
So I can say that and then it will think. And be like, wait before I do this, I need your approval, yes or no. So I can respond back, yes or no. So I'll say yes in this example. And then, cool, it'll go generate an image of a fluffy cat.

[00:07:54]
And then hopefully, this is a fluffy cat [LAUGH] DALLE 3 is wild. It's unhinged. But, if you get a good prompt, it's actually really good. So, let's see what image this thing generates here. There we go. It says, there we go, okay, fluffy cat. It's pretty good. So, if we run that back and I say, generate me an image of a fluffy dog, but because I'm a cat person, I'm gonna say, no, don't do that.

[00:08:29]
I only want cats. And then it should not do that. It'll process my approval. And then it should say, yeah, sorry, couldn't do that. Let me know if you wanna do it again next time. So we're gonna build a flow like that, which I think is incredibly powerful.

[00:08:45]
If your agents, I don't know, trading money out of someone's bank account, you probably wanna get their approval first, or something like that. So we'll be doing things like that.