Learning Multipliers with Larger Sample

[00:00:00]
>> Will Sentance: Figuring out or learning the multipliers for all the pixels with a larger sample. There's our familiar smile at the top.
>> Will Sentance: Someone in this room doubted that that was a smile, no doubt anymore.
>> Will Sentance: Someone who should not be named doubted.
>> Will Sentance: There's a familiar, not smile.

[00:00:29]
There's that image we were using as validation originally, not smile, or the flat mouth. That's gonna now be a new image in our training, in a sample that we're gonna use to come up with our multipliers. That's not a smile. And then we've got another very cheery smile, also undoubtedly, just squint.

[00:00:49]
And I promise, you're gonna see.
>> Will Sentance: That's a smile. All right, excellent, smile not, not smile. There's our new data set. We need a process to figure out the multipliers that convert as many of these pixels in the sample to match their associated label as possible, once our sample is too big to see.

[00:01:13]
Maybe you can look at these four and figure out the multipliers that work, but honestly already, I'm pretty impressed if you can. And go any bigger than this? You definitely can't. We need a technique, and we can't brute force all the multipliers, try every possible combination. Because, suppose we only had the chars that have -1, 0, and 1, and then there's 12.

[00:01:34]
That's actually not impossibly large. What's the math for that? How many combinations are there for that?
>> Speaker 2: 1,000.
>> Will Sentance: Is it? That was for 0s and 1s, you've got an extra bit here. So it's already bigger than that. Yeah, so it's 3 to the power of 12, if anyone wants to tell me what that is, but it's not that massively big.

[00:01:54]
>> Speaker 3: 530,000-ish.
>> Will Sentance: That's half a million combinations of -1, 0 and 1 for our weights. Already that size. And you can therefore see, and we've got to then apply every combination of that to all the pixels to see, to all the samples, sorry, to see whether x combination converts more or less of them to their target.

[00:02:19]
Let's have thumbs on that idea. I don't get what you're saying, I'm clear clarifications on. I need to have a set of multipliers here. We've got these two together, it's worked for sample of two. We have a sample of four, we're gonna need to try in theory, every brute force, every single combination, -1 0 0 0 0 0 0 0 0 0 0, -1 -1 0 0 0 0, -1 -1 -1 0 0 0 0, every combination.

[00:02:46]
There's actually half a million for just the value of -1, 0, or 1 across these 12. There's half a million permutations. Thumbs on whether, and we'd have to apply every one of those to our sample to see, does it convert to 1, -1, and then for our next two, which we're gonna do in a second, respectively, -1 and 1.

[00:03:11]
And you might be like, well, I can quickly figure it out, sure. But in theory, we could brute force, try every single one and find which ones do convert. Let's add our two extra sample elements. Help me out. No, maybe, I'll just write that up. I don't wanna make you all shout out one and zero again.

[00:03:25]
1 0 1 0 0 0 0 0 0 1 1 1, is this, Nick, a smiley face or a -1, not?
>> Nick: -1, not.
>> Will Sentance: Yeah, -1, it's not. Exactly, -1. We can see that when we draw
>> Will Sentance: On it's not, there we go. It's like a frog.

[00:03:54]
And then our final one we will have 1 0 1 0 0 0 1 0 1 0 1 0, look how happy. That's the happiest face I've ever seen. So is this Smile, Nick, 1 or -1?
>> Nick: 1.
>> Will Sentance: 1, this is a smile. Okay, we now need to figure out how to convert another set of 12 pixels, and their position, it's not just 12 pixels, it's 12 pixels in their position, pixel is number in position, right?

[00:04:34]
It's not just numbers, it's numbers in a position. This pixel matters, the number is 1 here, not here. And then the same thing for this one. We know multipliers now do work for all four. Our target output is respectively, 1 -1, -1, 1. Our accuracy, Maria, is out of what now?

[00:04:54]
Our accuracy is out of what?
>> Maria: 4.
>> Will Sentance: 4, what on? Sweet, brilliant. Okay, so, how the heck are we gonna come up with the weights? I know we came up with them well for here, but it's not gonna work. I can't eyeball this even before. Maybe you can, certainly not for a sample of a hundred thousand or even a million.

[00:05:12]
Well, we don't have a million images here, 3 by 4, it's only got an option of 4,096. But any bigger set of images, any bigger pixel set, 10 by 10, and you got a lot of samples. So what do we do? Here's what we do. We start with 0s for all 12 multipliers, all 12 weights.

[00:05:34]
We start with 0s. It's pretty clever how this, we're gonna see. Start with 0s for all of them in our matrix and our grid of numbers. And then we, one by one, compute for each multiplier, each weight. Is it better to tweak it up slightly or down slightly to convert in doing so more of our images to, that's the 12 pixels, to our correct label?

[00:06:06]
>> Will Sentance: And then something wild is gonna allow us to change them all at once. And hopefully get closer to more of our, or all of our sample eventually, being converted correctly between the 12 pixel values and the label. The larger the sample, the more conversion we get, the better it's gonna work on the rest of our population, we hope.

[00:06:30]
Not 2 out of 4,096, but maybe hundreds out of 4,096. Not two out of, let's say, a million sample, for larger images, but maybe 100, 000 in our sample. That's the only way we're gonna be able to do that, is if we can find a shortcut, an optimization to figuring out the best set of multipliers that convert as many of this to this, this to this, this to this, this to this.

[00:06:58]
So right now, our maximum could be four out of four. But for any large sample, it could be 80,000 out of 100,000. And we're gonna run these 12 numbers on all 100,000 images and see how many of them, the total comes to the total we want. We wanna have 80%, maybe 90% of them coming to the total we want.

[00:07:20]
For our sample of four, we want probably four out of four. Four out of four would be good, for our sample of four. But for a larger sample, 90% maybe. Okay, so let's do it. So multipliers of 0, what's our converted output for the first image here, Kayla?

[00:07:37]
All the multipliers are 0, so what's our total for our converted output, Kayla?
>> Kayla: 0.
>> Will Sentance: 0, but the next one, what's our converted output for the next one, Kayla?
>> Kayla: 0.
>> Will Sentance: Okay, yeah. All right, and the next one?
>> Kayla: 0.
>> Will Sentance: Smiley face. But the next one?

[00:07:57]
>> Kayla: [LAUGH] 0.
>> Will Sentance: Okay, all right, so it's all right. So our accuracy currently is, out of four?
>> Kayla: 0.
>> Will Sentance: 0, beautiful.
>> Kayla: I was trained.
>> Will Sentance: [LAUGH] Thank you, exactly well. Nice machine learning, pun, machine learning joke, there. Okay, now, very good. Now, accuracy's here out of four.

[00:08:18]
We've got to start somewhere. We start with 0s, okay? Now, let's try jolting this one up. Let's try going up with this one to a 1. I don't think we're gonna need to now do our calculations on the board in full, and say, 00, but rather, we can have a look.

[00:08:38]
Let's go around. Jamie, can you tell me what the output of, with a 1 here, we had 0 for all 0s, what's the output of the first image gonna be now?
>> Jamie: 1.
>> Will Sentance: 1, spot on, exactly. Thank you. 1, so that's an improvement. That's better than it was conversion.

[00:08:58]
And for this one, the output's gonna still be 0. So that's no change, no better. For this one?
>> Jamie: 1.
>> Will Sentance: 1, that's actually worse, right? It's got further away from the target, which was -1. It got worse. This one, the output, Jamie, is gonna be?
>> Jamie: 1.

[00:09:20]
>> Will Sentance: 1, which is better. So, that's kind of fun, tweaking up the lower middle is going to make how many better?
>> Jamie: 2.
>> Will Sentance: 2, how many worse?
>> Jamie: 1.
>> Will Sentance: And how many no different?
>> Jamie: 1.
>> Will Sentance: So accuracy, I would say, roughly, has got? Better. Better, overall, exactly.

[00:09:43]
Our accuracy has gone up across the full sample. So, I think we can confidently say, if we increase the value of the multiplier in the bottom middle, which we also remember, right, it was a 1 when we did it for our other side. So we are not surprised by this, per say.

[00:10:01]
Then we know the direction to go on that multiplier. Okay, now we get rid of it all. Forget it ever happened. Forget it ever happened and we do it all over again. We forget it ever happened, and that's crucial people, we do it all over again. But now, 4, that's now 0 again.

[00:10:26]
To our totals again, now, everybody, are what? 0, 0, 0, 0, and now let's try tweaking this bottom right up again. Well, let's try it again. Okay, top one, Ben. Output is gonna be with this new all 0s and 1s. Just this one is gonna be 1, 1 on.

[00:10:48]
Okay, better, better. There goes my American accent. This one here, output's gonna be?
>> Ben: 1.
>> Will Sentance: 1, uh-oh. Better or worse, Ben?
>> Ben: Worse.
>> Will Sentance: Worse, output of this one gonna be 1 1. 1, better or worse, everybody?
>> Speaker 2: Worse.
>> Will Sentance: Worse, well done. Output of this one, still gonna be 0, so no change.

[00:11:13]
Okay, have we improved our overall accuracy? Is it better overall or worse overall with our tweaking up the bottom right one?
>> Speaker 2: Worse.
>> Will Sentance: Worse, well done, people. So going up, worse, going down, better. We can show it, but going down, better. If we went to, I suppose we could do it, couldn't for a second.

[00:11:35]
We'd have a -1 here, we'd have a -1 here, we'd have a -1 here, -1 here, 0 here, and this one be worse. We'd have two better, one worse. So going down here, better. And then we get rid of it all again, and go back to 0 all over again.

[00:11:55]
Anyone wanna do it again? Okay, let me show-
>> Jamie: So we can deduce the bottom left corner is also better, down?
>> Will Sentance: Yes. Yes, is Jamie right, everyone? Yeah, well done. Good deduction. And then, actually, this one's better, down, this one's better, up, this one's better, down. And then these ones, roughly, no change.

[00:12:17]
So, Jamie was spot on there. When we could have kept going and checked this one as well, if you wanna do that. [LAUGH] But if you check each of them, but remember each time, get back to 0. It starts all over again. Forget you ever did the other ones.

[00:12:34]
Start all over again, go from 0. In this case, go from what it is right now.