Wrapping Up

[00:00:00]
>> Scott Moss:We've reached the finale. There are some other things that I want to leave you with. If you're going to use a framework, highly recommend the OpenAI Agents SDK. This is the one that I recommend. Let's go to the JavaScript one. I recommend this one if you're going to make an agent. So you know how to do all the stuff we did from scratch. Highly recommend the OpenAI Agents SDK. Everything that we just did here, it's done.

[00:00:31]
Those three short lines, done. Easy. That's it. That's all you got to do. You make a new agent, give it a name, give it some instructions, they give you a run function very similar to what we did. You pass on your agent, you give it a thing, you're done, okay? Tools, very similar to what we did. You give it some tools. The tools look exactly like what we have. You have name, description, they have parameters instead of input schema.

[00:00:58]
It's the same thing. It's still a Zod schema. They have an execute function. It's literally the same thing. You got your tools. Non-strict ones, whatever you want. You want to orchestrate agents, put multiple agents together, they have that. For instance, handoffs. You want to hand off something to another agent, you can totally do that. Use agent.create to ensure the final output type considers handoff, so you can hand off something else to a refund agent in this case.

[00:01:29]
So they have all that, they have context management, that what we have to build from scratch, they got all of that. Sessions models, they got something called guard rails that I was going to do, but it's not really that important for what we were doing, but you can have guard rails that check the input using an LLM to determine if this is something that you want to run. So like based on this input, I'm not even going to send this to the agent because this sounds toxic or this is, you know, it's not safe.

[00:02:01]
We're a kids agent app and you're talking about non-kid stuff, right? Like that's a guard rail. Human-in-the-loop approvals, you want to add an approval here, you just put needs approval on the tool, but done, and you could get approvals, right? So like all the stuff that we just built, this thing has it built in. And it's OpenAI. They have like a GUI that I showed earlier where you can attach this to.

[00:02:22]
There's some sophisticated things you can do with this. So that's one. I would say if I was going to use an agent framework, which I rarely do, I would at least try this one. The next one would be Mastra. This is like a new kid on the block, but slowly all they do is redo their website. That's really cool. They must have just did this. This was pretty cool too, I would say. This is the one that everyone is trying, this is from the creators of a little framework called Gatsby, if you've ever heard of that.

[00:02:54]
They're doing this now, but this one's really good as well. Can I go to their docs, please? It's also very similar. I think the thing that's cool about Mastra is they don't just do agents, they also do workflows, which is kind of cool. So for instance, if you want to make a workflow, you could do something like this where you can have a step, and then have another step, you can create a workflow and be like, cool, do this thing and then do this step and then when we're finally done, we can commit it.

[00:03:34]
So there's just control flow inside of this workflow where you can control what steps run and then depending on the output of one step do this other step. So it's like a workflow, right? So you could do that in code. So I think that's pretty cool. You know, they have guard rails, they have all this other stuff and the additional benefit of Mastra, I'm not sure if they have it yet, but they also have a hosting platform which you can just deploy directly to them, which gives you all types of depth tools and different stuff.

[00:04:01]
I haven't tried it. I don't know, but it seems cool, so you check that one out. I just learned about this other one, Volt Agents, like three weeks ago. Seems cool. It seems very similar to my idea of having agents that respond to events, not just prompts, so like something new came in on Slack, something happened on GitHub, something happened in your Airtable, let the agent handle it, right, versus only responding to prompts, and it being event-based, which I believe is the only really real way to make agents outside of agents that live in a terminal are event-based agents in my opinion.

[00:04:36]
But again, you can see they're all very similar. You make an agent, give a name, description, a model, and some tools. It's very similar. It's not nothing crazy, right? So that, and then other things I want to leave you with are just some really cool tools and agents that you might want to check out. So check out Browser Base. This is basically give your agent a browser. They have different products, the ones that are going to be super easy if you use Browser Base.

[00:05:07]
It's essentially like headless browsers as a service. If you use Stagehand, it's actually an AI agent that can use a browser off of human language, right? So it uses Playwright, but it generates Playwright code and then runs it based off human language. So it's kind of cool. I actually, one of the first agents I built was this, and it was before Playwright. Theirs is way better than mine, but it was something that I was trying to build for my startup, and then like around the same time, like everybody started building them, so it was unfortunate, but these folks know what they're doing.

[00:05:41]
I've looked at it, it's pretty impressive. The other one that's really great, it's not a tool, but it's a product. It's called Director, so if I go here and I say find me some LeBron's on Nike, size 14. I didn't spell Nike right, but let's see what happens. What this should do, let's see. Ooh, they changed their UI. It should open up a browser for me. Usually a browser would pop up and you would see it using the browser because that's actually what it's what's happening right now.

[00:06:23]
They're just not showing me anymore. Oh, here we go, yeah. So they have like a browser and you can see all the steps that it's taking into the browser. I think it's going to nike.com. Yeah, it messed up. Yeah, I was going to see if it was going to fix it, but yeah. And you can see all of what it's done. It's a helpful tool if you wanted to like, you know, crawl a browser or whatever. What else is cool?

[00:06:47]
The only thing I could think of is lately I've been deploying all my agents to Cloudflare. They are not sponsoring me. I could care less about Cloudflare or any deployment company, but when it comes to agents, they're just killing it in my opinion, because they have every single thing that you need to make an agent, they even have an agent abstraction. It's not quite a framework, it's like a combination of infrastructure and abstractions as far as like state workflows, durable execution, WebSockets for free, streaming for free.

[00:07:22]
It's fast because it's V8 isolates on the edge. This is going to be my go-to platform for the foreseeable future for making agents because everything that I need is in one place outside of like a Postgres database, it just has everything. Whereas if I go somewhere else, I got to pull in all these other things. So I've been used to Cloudflare, they have Cloudflare Agents that are pretty good, so I would, they have like starter kits, highly recommend diving into it, giving it a shot.

[00:07:54]
I don't think anybody is on par with them, in my opinion. Not for the types of agents that I see, like background agents that do events. Nobody's close, in my opinion. So I think that's all I got. There's probably more, but that's all I can think of right now. Any questions for me? Any topics you wanted me to talk about? Anything to explore? Yes. Which skills do you think are essential for future career success in terms of LLMs and agency?

[00:08:24]
I'm not even going to answer the question, I'm just going to show you, so let's, I'll let you decide. I'll just show you something. I'm going to go to anthropic.com, okay? I'm going to go down here. I look for careers, yep. This is how you determine the future. You go look at the people who are hiring. I'm going to click on AI and research engineering, okay? I'm going to search for evals. Okay, research engineer, model evaluations.

[00:08:52]
I'm going to click apply. San Francisco, New York. Before we even look at any of this stuff, I'm going to see what are the qualifications. Nowhere in here does it say a master's degree or PhD. It just says experience with evaluation during model training, particularly in production environments, familiarity with safety evaluation frameworks and red teaming methodologies, background in, what was that?

[00:09:20]
Psychometrics? Experimental psych, remember I said psychology? Psychology, experiments with reinforcement learning, contributions to open source, knowledge of prompt engineering, experience managing evaluation infrastructure. Okay, 300K to 400K. Okay? There's no degree requirements in here. You just got to be a psycho, apparently. So, you know, there you go. There's one. Let's see what else they have in here.

[00:09:45]
I thought there was another eval one in here. These are all research engineers, so another one that might be very, oh, here we go. Here's a founding design engineer. This is someone who's like really likes to work on the front part of front-end engineering. I'm talking like motion animations and stuff. Yeah, that's what they're hiring for. Where is it? They didn't want to put the salary on this one.

[00:10:13]
Hold on. They're like, wait, wait. Oh, there it is. Yeah, 320 to 405. It's even more than the research one. It's insane. Okay, so I do believe there's still a high need for people who can build delightful experiences because the problem with AI right now is UX and UI, it's all bad. So if you understand that, that's good, but also evals, you know, anything like production type stuff is going to be good.

[00:10:49]
Let's see, what else? I mean, if you want to get into research, you can get into research. Like if you already have a degree in computer science, highly recommend maybe getting into publishing white papers or co-authoring white papers, find a bunch of research scientists that you could work with and help them set up their experiments and you can start by looking at white papers and seeing if you can replicate the results that they're getting on some rented infrastructure, because if you can get into research, I mean, you got Meta throwing around hundreds of millions of dollars just to hire one person, so like the limit is very high.

[00:11:28]
I'm not saying that's where you'll be, but the limit is high. There's nowhere near enough research scientists, research engineers out there, so highly recommend getting into that. And at some point I was going to go do research, but for my own reasons I decided not to. So, yeah, I would say anything research related, AI engineering in general. It's a new term, engineering. AI engineering is essentially what we've been doing in this course, which is like the process of building production-ready AI agents, not so much on the research side, it's the implementation side.

[00:12:05]
There are tons of those all over the place, they all pay well over six figures, well, they're six figures but well over 100K I should say. That is a very good skill set. Like if you can make an agent, like, if you understand what we just went through for this course and how all these things work and the trade-offs and stuff like that and you can specialize in one or two of those things, I highly recommend evals, and maybe even like the infrastructure side, I think is really important.

[00:12:31]
If there were two things I would master, it'd be evals and infrastructure side. Everything else is mostly solved. I would do that. So yeah, if I was, if I was learning right now, if I was three years or less of experience, I would drop everything and learn that stuff. I wouldn't learn anything else. That's just my opinion. There were two websites you looked at. One was the sandbox.cloudflare.com, and the other was E2B and Daytona.

[00:13:04]
What's that? E2B and then Daytona. No, I was thinking Browser Base. It's not related to the function, but I'm just looking at the website and I'm wondering, like the, like just blocky style, if there's a name for that. I feel like brutalism is that, is that what they call it? This is semi brutalism. Okay, I mean, I just, I love the look of those websites and I don't know if that's common, but since I just saw two of them today, yeah, it's semi brutalism.

[00:13:32]
It's not full brutalism, but yeah, I would call that brutalism. I'm going to be looking into that. Yeah, yeah, it's cool. I like that design language too. Yeah. Okay, well, thanks for taking the course, learning how to build AI agents. I challenge you to go off and either continue building on this, learn from the code that you didn't write that was already in there, or go make your own, like, experiment with building and stuff.

[00:13:56]
There's going to be a massive opportunity I think for quite some time for folks with this skill set to benefit from and if you're already a successful, well-skilled engineer, like being able to upskill to this won't be that difficult. And also if you're somewhat new to engineering, you're going to have to grind anyway, you might as well grind on this. That's just my opinion. I mean, there's also other things as well.

[00:14:20]
I just think the competition for those other ones are going to be very difficult, as you start to see more companies invest more into like, well, how do we make an agent or what do we do here? Like that person in your company that knows the most is probably going to be doing the best. They're going to step up and like, well, I know the most here, so I'll do this. That person's going to be doing well very soon.

[00:14:40]
So that's my suggestion. Get in there, make some agents, go learn more about the different types of agents, follow Anthropic, follow OpenAI, follow all these research labs, look at the white papers that come out, look at the frameworks that are coming out, try to use some of these things, just experiment, get lost, have fun. At the end of the day, the more lost you get, the more questions you have, the more knowledgeable you'll be.

[00:15:01]
I promise you, like you, like, you think I know this stuff, but I have way more questions than I have to share with you. Like there are so many things that I don't know. And it's those open questions that drive me to keep going to be like, I really want to know how that works, or I really want to know how they came up with that, or I really want to know if I'm capable of implementing that. So I have so many open questions, right?

[00:00:00]
The day that I don't have questions is the day that I stop learning. So go get lost, get those questions and, you know, upskill yourself.