Zero Shot Classification

[00:00:00]
>> Steve Kinney: There's a whole bunch of little things that we can tweak and we'll play around with at various points. But yeah, I would say like play around with some of the different ones, go up to 1.2 or 1.5 as well and get a sense of what you want to come out of it, right?

[00:00:14]
And with all things, right? Because again, this is simply a statistical model that goes based on the previous words. All previous words right in the prompt, go ahead and generate the next few clearly like the words that you feed, it also play a role beyond the knobs that you can turn.

[00:00:33]
And so if you start speaking in a formal style, you are likely, because if it's trained on, read a bunch of academic papers and a sentence starts out in that kind of academic speak, statistically speaking, the next most likely word will be that so on and so forth.

[00:00:51]
So why don't you take a few minutes and go, I think play with a bunch of those numbers or any of these parameters and let me know in, I don't know, five minutes if there's anything that was particularly interesting or fun or anything along those lines. So the other one that I said was pretty cool, came to that when we talked about sentiment analysis is this idea of zero shot classification.

[00:01:10]
Zero shot means if you think about when you have a conversation with ChatGPT, that first prompt, if you got what you want, is effectively zero shot. You said one thing, you got result beyond that, then you're doing a lot of other techniques but sending one thing and getting the results.

[00:01:32]
What's cool about this is, like I said before, you send one thing, it does not necessarily even need to particularly be trained. You say, hey, I've got this string of text, hey, I've got these categories. Why don't you figure out what label it should have? There's no fine tuning at this point.

[00:01:49]
There's no special tweaking of the model. It's just using some of the stuff that I haven't explained to you yet because I felt like lecturing to you for three hours about philosophical conceptual concepts before showing you any code was rude. I'm just trying to get you excited. We'll see how it works.

[00:02:07]
You can kinda give it both and it will figure out those things effectively. Saying this text is about a given category and then it will see how well the hypothesis fits and it will give you a score in this case. So here we'll scroll back up to the second.

[00:02:21]
Let's actually just look at it. So here we've got yet another pipeline Again, the tasting note is for all of these especially particularly simple small kind of model things. I mean, text generation, one would argue, is the big thing that we all use every day in the rest of these models.

[00:02:39]
So they can be small, tiny models. They can also the big things are just the little things at scale, right? If you took one of these small models and fed it all of the world's Internet and gave it like a giant server farm, you too could turn these small models into that as well.

[00:02:58]
But considering we are running on commodity hardware at this point, not bad. So yeah, we just pull in a pipeline for this zero shot classification. The default model in this case is from Facebook and I just arbitrarily came up with a bunch of candidate labels. Again, it is not trained on these.

[00:03:19]
You could theoretically take data from your logs, data from something else that you have, grab one of those data sets, like I said on hugging face that look interesting and theoretically begin to classify it. There's all sorts of data sets. You can go look around, it's fun. But here I was not feeling particularly creative that day, so I think I was angry about the Lakers at that moment, who knows?

[00:03:50]
And you know, you can come up with a bunch of categories and not being trained on either the categories or the strings. Again, it comes up with a hypothesis. It then uses the attention stuff, which I have not talked about yet again, building up anticipation to figure out how well those things fit in and then kind of figures out what label is most likely for each of these, right?

[00:04:12]
And again, part of the game here was choosing strings and categories that were at least obvious to you. So like, cool, he put in random stuff from his logs or whatever or some data set I've never seen and some labels that I don't know about. Yep, definitely. But like, ideally you should be able to squint at each of these and probably figure out what one they apply to specifically, because I'm clearly not that creative.

[00:04:40]
And I'm pretty sure the strings are in the same order as the candidate labels. If you don't believe me that this works, you can shuffle them as well and get a sense of that one too. So here we again will just iterate over the text and we'll just pass them into that classifier pipeline that we created.

[00:04:56]
What we want to do is then see for any given one what. What label should it have, right? And again, where things might become the very practical thing and I worked at an email company for a little bit and both in terms of accounts and also emails. Is this spam or not?

[00:05:22]
Is this account, you know, like, you know, information, like in terms of like payment information, probably fraudulent or not. So there's a lot of like this one's got a lot of practical use cases, right? And you can see that I'm not gonna make you do this math, right?

[00:05:42]
But if you're willing to believe me, guess what? All of those decimals add up to the number one, right? And so it will theoretically kinda show you the different weighting of what it believes that it is. And again, the reason I chose, kind of silly. Silly, but whatever.

[00:06:05]
Strings, which is like Apple reported record quarterly earnings driven by strong iPhone sales. It's like that seems to be. We're two-thirds pretty sure it's about technology. One third sure it's about business. Yeah, I can agree with that. I can sign on to that newsletter. The artificial intelligence model shows significant improvements in natural language understanding that is most likely about technology.

[00:06:33]
Yeah, me about the other ones, which again, having chosen strings that were supposed to get certain classifications, I feel good about it. Okay, this is predominantly about sports, a little bit maybe about entertainment, but as you can see, barely. So again, not knowing about any of those categories beforehand or any of the strings and in particular, while I'm talking.

[00:06:59]
Go ahead. You can edit any of the code in here and run it, try out some different strings, add a few more strings, add a new category, get rid of some categories and see what happens. The cool thing about this is you can experiment with all the code that I have on screen as we go through and try out different things.

[00:07:16]
If you have some data you want to put in there, go paste it in. These are all totally interactive. You can just hit the play button at any given point and rerun that cell then and get the data as well, right? So like I said, it will. Without any kind of magic.

[00:07:33]
None of this is magic. But without any kind of work. How about that? Without any kind of work on our end to get the results, it does do a pretty good job of taking arbitrary strings it's never seen before and arbitrary categories it's never seen before and try to figure out the relation.

[00:07:45]
And again, for these tiny models, right? Did you need to pull in and start paying the cost per token for a giant world class foundational model to do some of these simple things or can you grab things that you can fit very easily in a vm? Some of these you probably get done serverless if one put their mind to it, right, and build some features out that would have taken you a long time otherwise.

[00:08:13]
Or even worse, write your own algorithm. There are some pitfalls, though. If you give it 100 categories and all those answers add up to one, it's gonna end the way you think it's going to end. Everyone's nodding cuz yeah, that's what's going to happen. So you gotta keep that in mind.

[00:08:35]
And then having categories that are too similar, there are things that you need to tweak and consider, so on and so forth. And also, and this becomes interesting, if you have wider amounts of data, what do you do which is like, do you need to go grab a different model if you're processing Spanish, so on and so forth, do you need a layer beforehand?

[00:08:54]
Either one of the translation models or something along those lines? Do you want to run through a translation model and then pass it to an English classifier? Maybe do you want to at least detect what language it is and then have different models that you slot in there?

[00:09:14]
Because you could theoretically, the model is just an argument. You could theoretically have many a classifier, like, you know, a key value store for the different languages and then grab the right classifier, right? These are all where the engineering part come in of like, for the unique specifics of your problem.

[00:09:33]
What do you do, right? Welcome to the fact that none of this solves anything for you. Pitfalls are just. Yeah, again, doing any of the things that you think would be probably somewhat bad, right? But yeah, routing tickets to departments, content moderation, so on and so forth. The one that was near and dear to my heart that my buddy Steven worked on.

[00:09:53]
And again to the question that you asked earlier about various emotions beyond positive, negative. This would probably be a better solution than a model that, cuz then you could figure out which emotions you care about. [LAUGH] Right. Are we going to go the full. What's the Pixar movie where they're all different emotions?

[00:10:12]
>> Student: Inside Out.
>> Steve Kinney: Inside Out, yeah. Right, you just pick the inside out characters, right. Or like, honestly, considering that you can just run this cell, I don't know what Power Ranger it aligns to. What are the creative use cases here? Which Ninja Turtle does this most identify with?

[00:10:29]
That's brand new content. I haven't thought about it. I might try that out later. You can also try it out. Any of these you should play around with as well. You can use multiple labels too. I think I'm using. Am I using a slightly bigger model this time?

[00:10:43]
What did past me decide? Nope, same model. So in this case, if we wanted to even have a version of sentiment analysis that was positive, negative, neutral. We can do that as well. Let's go take a look at what this looks like. Cool, so did we get something good?

[00:11:08]
>> Student: I changed all the categories to happy, sad, indifferent, angry and mean. Just looking at how it came out.
>> Steve Kinney: What did the new iPhone come out?
>> Student: That was the reported quality. The iPhone says that was happy, was 0.7. Yeah, right.
>> Steve Kinney: Apparently Facebook's model is happy about the new iPhone.

[00:11:34]
One of the interesting tasting notes about the multi label classification is you can see in this case it is that point on a 0 to 1 scale. At that point you're sorting by the highest number. These do not add up to one, as you can see very quickly.

[00:11:50]
And so it's very confident that it's about technology, pretty confident about business, and half confident that it's a review. And so those don't add up to 1, but they are at least between 0 and 1. And you could theoretically sort of and pick the other one. Let's comment that one out so I can do find out what this model's feelings on Python is.

[00:12:20]
It's like pretty sure I love programming Python as positive. It's not entirely sure. It could be neutral, but it's definitely probably not negative. So you can kinda play around with these and try out different categories and that's the kind of the fun piece of what other data do you have you could figure out?

[00:12:35]
Could you take a bunch of strings from books and figure out is this fantasy, science fiction, fiction, nonfiction, historical fiction, so on and so forth, right? Like obviously there are a ton of interesting things that you can do. And like I said before, some of this becomes, well, grab a data set that I grabbed, but I'll show you where to grab other ones.

[00:12:54]
You can grab tons of data sets, all sorts of different things and play around with them as well. Yeah, like I said, as we saw before, you can pull in categories like happy, sad, angry, surprised, even try out some longer labels as well and kind of get a sense for how these things work.

[00:13:17]
And again, I chose the default small model so that as we were talking, we didn't have to wait for one to download for 10 minutes. But obviously if you're like building something or even whether it's a hobby project or a real deal project, 10 minutes is not going to kill you.

[00:13:32]
It's only going to kill you when you're watching me stand at a podium. So some of the larger models I think become interesting in that sense as well. But yeah, definitely try out different things. Changing the strings, play around with stuff. I think that's where the interesting pieces lie in this case.