Microsoft AI Toolkit Extension

[00:00:00]
>> Steve Kinney: And then one more on that theme, and we'll scroll through here in a second. That is super cool, and this one is from Microsoft, it is called AI Toolkit, which is really useful for developing stuff on top of models, right? Let's take a tour of AI toolkit because it is very cool, and here's some other git lens stuff that happened after I installed that one.

[00:00:25]
But let's take a look at this right now. So if I look on the side,
>> Steve Kinney: You can see you can have access to a bunch of different models that are either hosted, like these are not ones you necessarily have to even download, they are hosted by various providers.

[00:00:42]
You can see that some of them are hosted by GitHub, some of the hosts are OpenAI, some of the hosts are Anthropic, so on and so forth, right? I added in Claude 3.7 and OpenAI GPT 4.0. For these, you need to have an API key that's about it, the GitHub ones, you don't always need an API key.

[00:01:02]
It just depends on which service you use, it will tell you and prompt you for one of them if you're supposed to have one. And you can even fine tune models, not that I have the GPU for that on this computer. But where I think it becomes really powerful is there is stuff like in the playground, you can compare two models, right?

[00:01:26]
So all of these different models, especially the new ones that come out, have different things that they are good at or not good at. So in this case, I've got Claude on the left side, and I've got GPT-4.0 on the right side. And we can do is if you are trying to develop a workflow on top of one of these models, you can go like, and you can see that I was typing in both of them.

[00:01:53]
How many moons does Saturn have? And it will run it through both of them, and you can kind of see which, depending on what you're trying to do, what has the better output for you in this case. So you can kinda see the output across two different models and compare them.

[00:02:12]
So if you are trying to figure out, because a lot of times, obviously, I grabbed two fairly large general-purpose models. But if you wanted to compare, hey, a very small lightweight open-source model, is that good enough for the task I'm trying to do? Right, you can at least compare versus opening up 9,000 tabs and pasting the same thing in.

[00:02:34]
You can kind of experiment with different prompts to see what works. And a whole bunch of other stuff that you can do in this case, you can give it instructions. There's the ability, one of the really powerful things with a lot of these models is the typical use case for a lot of people is just the chatbot, right?

[00:02:53]
But for almost all of them, these days, you can also give them a JSON schema and say, okay, respond in this JSON format, right? And make sure it's valid to this schema. So then you can work with it programmatically, which is really powerful for taking unstructured data that you would normally set to sit and write a parser for, and getting it back as structured JSON, right?

[00:03:15]
And you can kind of do a lot of that, not necessarily, when we think about Copilot or even the ChatGPt plugin, we think about the AIs are writing code for us. This is kind of more the opposite, which is if you are building some kind of workflow, some kind of feature on top of one of these things, you can now compare and contrast the different models.

[00:03:36]
There's a bunch of other really powerful features in here too. So you've got this prompt builder where you can kind of, run it in this way, but you could also, again, change it to the JSON schema that you would like to build. If you have a local JSON schema, you can just load it right on up.

[00:03:55]
Things that you can only do with the API normally that you can't do in either like the cloud UI or the ChatGPT UI where it's, okay, I need the responses, not to be just some pros and your thoughts and hot takes, I want them in the structure JSON, so on and so forth.

[00:04:11]
You can just find a JSON schema file. That JSON schema is just a JSON file that defines a Typescript type basically. And load that all in, in this case, use an example, and where you wanna save it. So in this case, I can run it, and I will save it out to a JSON file.

[00:04:34]
And you can see in this case, instead of getting some pros, I get the steps. And now I could theoretically in TypeScript or JavaScript, or honestly any language, because it's JSON, work with this data structure instead of just a wad of when it gets creative and thinks it's funny.

[00:04:50]
And set the system prompts, so on and so forth. The other really useful thing is if you have a CSV full of data, maybe a bunch of values, and you wanna just create 100 prompts with all of that, or that's gonna be expensive, either on time or money.

[00:05:11]
And send out a batch one where it substitutes the values across like that where you're not doing the same prompt ten times, right? You can either, in this case, generate a data set or select a CSV or something along those lines, generate a whole bunch of prompts, again, text or JSON schema, so on and so forth.

[00:05:29]
Set a whole bunch of parameters that are not available outside of the API. And again, begin to develop software on top of these things. And the important part is I kind of alluded to earlier, this is just an extension, but you look at this, this is a full on almost app inside of VS Code, right?

[00:05:48]
So when we think about, when we look at extensions later, there's everything from highlighting the word to do in red all the way to building a full-on app experience inside of VS Code. Again, there's you can create all sorts of evaluations of different models and test them and basically benchmark them.

[00:06:10]
If you are building software on top of these models, there is a lot of features and tools in here. I have personally not tried to fine tune a model using this cuzI don't, this little guy can handle that. But theoretically, if you had a more powerful machine, you could probably do that as well.

[00:06:27]
You can grab open-source models, a bunch of really interesting stuff on top of all of this, which is super cool. But I point this out for two reasons, one, it's cool, it's free, yay. The other part is, I also picked it to kind of show the extremes to how far you could take building a VS Code extension, right?

[00:06:45]
You could theoretically build an entire app inside of VS Code, right? Everything from a SQL, a visual SQL IDE or something along those lines, see all your SQLite tables for your app, whatever you happen to do. If there's tooling that you need, right? You could theoretically kind of take the foundation that's already here for you and integrate it in all sorts of interesting ways, so on and so forth.

[00:07:10]
You can see in this case, it will even generate the Python code for me, right? I don't think at this moment that it will generate the JavaScript code for me, but that's fine. Yeah, so there's a wide range.