OpenAI API & Tokens

[00:00:00]
>> ChatGPT works in this way. ChatGPT is in the middle, is that logo we have there. ChatGPT or any GPT model, actually, receives a prompt, and it gives a result, that's kind of the idea. And in this case, ChatGPT as an app is using a GPT model that looks like a black box there.

[00:00:22]
And the color is not random because it's actually a black box, we don't know what's inside. We don't know how it works. We will see this in action in a couple of minutes. But we treat that piece of software, let's say it's a software, as something that we don't actually understand what's inside.

[00:00:42]
The only thing we know is that we put a prompt and we receive something in return, okay? What's a prompt? A string, just text. What's the result? A string, another text. That's why it's known as a large language model. So that's the basic thing. And maybe if you're a developer, you're expecting this to be different later during the day in this course.

[00:01:11]
You're expecting me to show you an API where you can actually get into that black box to make things differently or things like that. And that doesn't exist, actually, that doesn't exist. So I got that as a surprise when I was trying to understand how this work. So GPT, we have different versions of GPT, and there will be more versions in the future.

[00:01:33]
That's the only thing that you pick, the version. And there are some settings as well, but we don't have an API for do something different with the model. It's always a prompt and a result, that's all. So we need to deal with that. That's why there is something known as prompt engineering.

[00:01:50]
Because the only input we have there is the prompt. So we need to get experts integrating that prompt. Well, when we are setting a prompt or we are sending a prompt to ChatGPT or to any API that is GPT-enabled, and we're getting the result, we are measuring those strings with something known as tokens.

[00:02:17]
Tokens will be the most important unit for us from now on, why? Because that's what we are paying for. A token is a sequence of characters, or, let's say, subwords that the model uses as the basic unit of processing and understanding natural language text. So we are not going to be charged in our credit card, we will get into that in a couple of minutes.

[00:02:46]
We are not going to get charged by the character or by the word or by the paragraph, we're going to get charged by the token. So that's how we work. So during the month, OpenAI will count how many tokens we've used. And you will use tokens when we prompt and when we receive the output, okay?

[00:03:08]
Yeah, you're actually in charge of the prompt, not of the output. But what you can do is you select the maximum amount of token that you are willing to pay for every prompt. So you will say, hey, I'm going to pay for only ten tokens, not more than that.

[00:03:26]
So we can narrow the output using that maximum tokens property that we will see where it is in a minute. And just to give you an idea of what a token is, in English, it's roughly 4.5, so between 3 and 6 letters per token. So one token, let's say it's four letters, but that's different per language.

[00:03:54]
And also that's different when you are applying source code. So because ChatGPT or GPT can also read and write source code. So in that case, it's not per character. For example, when you open curly braces, that might be just one token. So it's not counting really characters. And when you're writing in different languages, Chinese, Korean, it's not like four characters will give you a token.

[00:04:18]
So it's a different way of understanding that. At this point, you might be thinking, how do I know how this works? Well, we do have in OpenAI a tokenizer tool. It's platform.openai.com/tokenizer, where you can write some text. And you can see how many tokens this particular text will get.

[00:04:47]
In this case, it's going to be three tokens, but I can just copy any text and see how many tokens it will take. You will see that it's not always the same. For example, here I have one comma, the comma is taking one token we have for the comma, okay?

[00:05:06]
But at least with this you can estimate based on the prompts that you will use and the results that you are expecting. You will try to understand roughly or to estimate roughly how this is gonna work. By the way, here we see GPT-3. GPT, we know what that is, it's the LLM model.

[00:05:26]
What about Codex? Codex is a different model that OpenAI, the company, is going to deprecate pretty soon. And Codex was actually optimized for source code. But because now GPT is pretty good at source code as well, they are deprecating Codex. It was a different model, so different models will calculate tokens in a different way.

[00:05:50]
So that's about tokens. Well, what about the price of tokens? Well, of course, pricing might change. So I'm just giving you the rough price as we are shooting this video. There is a pricing website at OpenAI where you can get the exact pricing. But just to give you an idea, ChatGPT, or GPT 3.5, that's actually the version that ChatGPT is using, cost around US$2 per million tokens, okay?

[00:06:24]
Yeah, you won't pay per million token, you will pay by the token. So if it's $0.01 cent, you will be charged $0.01, but that's actually the price. So GPT4, that is the latest version of the model available, that you may have heard that it's actually much better for some special tasks, such as coding and complex tasks, is actually 30 to 60 times more expensive than 3.5.

[00:06:58]
So that power comes with more processing power used, so that's why it's more expensive. The advantage is that it has a higher token window. Is it okay? What's that? What's a token window? So now we know what a token is, and we know that we will prompt a model and the model will respond with a string, okay, we know that.

[00:07:23]
And we know that that's expressing tokens. What we don't know as of right now is that models have a limit of how many tokens you can input, you can prompt, or receive. And GPT 4, it has a higher token window. If I wanna send GPT a book because I want GPT to summarize the book, I need to calculate how many tokens that book will contain.

[00:07:57]
And GPT 3 can maybe read one page per request, only one page. But ChatGPT 4 can read 60 pages, 80 pages of that same book. Of course, I will pay more, not just because the token is more expensive, but because, of course, I'm paying per token. But having that higher token window means that I can give more context to that LLM black box to get better answers, that's the idea.

[00:08:32]
Also, GPT 4 will have in the near future, it's not today available to every user, image and prompting. That is the idea of taking a picture and ask questions about the picture. So I can take a picture of the room here, and I can get an answer of, okay, how many chairs do I have here in this room, or ask questions about that picture.

[00:08:59]
That API was demoed by the CEO, but it's not yet available to everyone. When that happens, it will have its own pricing system. Also, there are other models available on OpenAI that we won't use today. But have in mind, it's not everything ChatGPT, there are other models, the most common one is called DaVinci.

[00:09:23]
Those models are not optimized for chat, for a conversation between a human and the LLM. It was more about completion, so you were actually saying I'm a web developer, and because of that my primary language is, and the model will give you JavaScript, probably, right? But it's not about answering questions, it's more completion, it's like the grandfather of ChatGPT.

[00:09:53]
Those models are still there and you can still use them. And sometimes they are cheaper than ChatGPT if you wanna use them. But they are for more precise specific situations. And we know that LLMs can be used for anything, so most people are just using ChatGPT. Also we have image models, and later today we will use one, DALL-E, actually.

[00:10:18]
So you can actually send a prompt, that's not an LLM, it's a different model. You will send a prompt like, I want a web developer that is actually thinking about an artificial intelligence. And you will get an image that was created based on your prompt. And for everything we have an API.

[00:10:38]
And we will use that API from our web app. And finally, there is audio models. We have that audio model from OpenAI open source. But if you don't wanna download and run that directly in your computer, you can use the service. In that case, the audio service they have is for transcription.

[00:10:59]
So we can get the audio of what I'm speaking right now. We can get really quickly, like three hours of audio, you can get the text transcript that works pretty well in around five seconds, that's how quickly. So three hours in five seconds. So those are the models that we have today on OpenAI.

[00:11:21]
What about how to create an account on OpenAI? First, it's not the same as having an account on ChatGPT. So you need to go and create a different account. You go to platform.openai.com. It's free to sign in, there is only one restriction, is that you need to validate your phone number.

[00:11:40]
So you need a valid phone number with an SMS, and that phone number shouldn't be a VoIP phone number. So it's not gonna work if you try a VoIP like an online number such as Skype or things like that, it's not gonna work, it should be a real SIM card.

[00:11:57]
And you will get, if this is the first time you will try this, and you can do that right now, you can pause the video and do that right now. If you sign in, you can get free credits, today it's $5 that will last 3 months. And in that case, you can start playing with the API and do all the exercises that we will be using today, not paying a dime.

[00:12:26]
So $5 will be good enough for everything that we will be doing today and more. If you are thinking I will create different accounts so I can get more than $5. Those $5 is per phone number, okay? That's why they're not accepting VoIP numbers. There will be rate limits as well.

[00:12:47]
This is in case you are getting really professional or your service is going viral, there are some rate limits. So you cannot get into one particular API more than 500 times per minute. For example, GPT 3.5, so that's ChatGPT, the cheaper version, it's 3500 requests per minute. It's actually pretty good, for a start, at least.

[00:13:16]
It depends on what you're doing, have in mind that you have that limit per account. And GPT 4, it's only 200 requests per minute, so it's more expensive, it's much better, but also it's much limited, today, at least. So this is important for those of you that are working for a company or creating products for companies.

[00:13:41]
When you're using the OpenAI API, all the interaction that you're having will not be used for training or improving models at OpenAI company. That's not the case for ChatGPT. When you're using ChatGPT, even if you're paying for the plus service, the data that you are submitting may be used later for training.

[00:14:07]
When you're using the API, that's not the case, it's part of the contract, okay? So legally, everything that you say there is not going to be stored or used to train other models. So this is important, and if you're a ChatGPT user, you probably know this. OpenAI GPT, both 3.5 and 4, the training data is actually up to 2021.

[00:14:38]
So they might not have knowledge of current events, so you shouldn't rely on OpenAI for events in the past two years, so have in mind that. That was the basic of OpenAI and understanding how that works. So at this point, the next step for you is to go and get an account, because without an account, you won't be able to use this.