Applying All Weight Changes

[00:00:01]
>> Will Sentance: And that tells us our first, well, let's see what it says, then make all of those changes up or down, let's say step of 1, step of 1, which is what we did to test it. Make all those changes at once. And you might be like, wait, wait, wait, and then start again with a new round of independent weight checks, independent checks.

[00:00:29]
So let's make all those changes at once. And then you might be like, excuse me, how am I possibly allowed to do that when I tested the impact of the direction by resetting everything to zero? And now I'm gonna do them all at once? That feels like I'm gonna be missing something.

[00:00:46]
There's gonna be some interaction, like if I raise this one without changing the others, that's very different, surely, isn't it? To raising this one, lowering this one, raising this one, how can I possibly be sure that doing all together, it's still better to raise this one? Well, we'll come to it in a second.

[00:01:06]
[INAUDIBLE]
>> Will Sentance: Maybe it's not, maybe it's not.
>> Male Speaker 1: Test it and see.
>> Will Sentance: Let's have a look. So, let's update all of our weights, then the top ones we said didn't change. Let's update all of our weights with these directions. So this one would go from 0 to 1.

[00:01:24]
What's this one, Nick?
>> Nick: 1.
>> Will Sentance: 1.
>> Nick: -1.
>> Will Sentance: -1.
>> Nick: 1.
>> Will Sentance: 1.
>> Nick: -1.
>> Will Sentance: -1.
>> Nick: 1.
>> Will Sentance: 1.
>> Nick: -1.
>> Will Sentance: Beautiful, now, our hope is we have managed to find a technique that didn't require us trying all possible combinations of these. But instead, just checked each of them individually to see if it overall improved the number that we're converting for just that tweak, leaving all at zero, forget it.

[00:01:55]
Change this, get them all back to 0. Try this one up or down. Forget everything else, try this one. And then we could do it all at once, even though that was surely not possible, because I know that it was the right move when I held everything else constant at 0, it made everything better.

[00:02:13]
But surely it's not the right move if I'm also changing a bunch of other stuff. Let's test this out and see what we get to as our new sums. Okay, now, people, we get to do some real good old arithmetic. And I need some help. I need some help here, people.

[00:02:33]
I need some help. I need some help cuz this is, okay, so these were our first steps. So we've done this now, we've used this. But some of the weighted pixels, the multipliers applied to our pixels, the top ones are all 0, right? Then this one here, 1 by 1 is?

[00:02:53]
>> Male Speaker 2: 1.
>> Will Sentance: 0 by -1 is?
>> Male Speaker 2: 0.
>> Will Sentance: 0, so what's our running total right now?
>> Male Speaker 2: 1.
>> Will Sentance: 1, plus 1 by 1 is?
>> Male Speaker 2: 2.
>> Will Sentance: Right, so our running total is?
>> Male Speaker 2: 2.
>> Will Sentance: 2.
>> Male Speaker 2: Yes, sir.
>> Will Sentance: Yeah, running total after the bottom left 1 where it's -1 by 1 is, so -1, now our running total is?

[00:03:10]
>> Male Speaker 2: Back to 1.
>> Will Sentance: 1, then our running total is?
>> Male Speaker 2: Up to 2.
>> Will Sentance: 2, and now our running total is back down to?
>> Male Speaker 2: Back to 1.
>> Will Sentance: 1, amazing. So our output is 1. Okay, yep. Next one, anyone got ahead of this and worked it out?

[00:03:24]
We've got three negatives and two positives, right? So our total is? -1, excellent. -1, does that equal -1? Yes, and for our next one, we have all of these accounts, sorry, we're doing, this was our getting there. I almost wanna delete this. No, I can leave it there, but these were the steps we took to get to this.

[00:03:51]
These were the steps we took, we worked out which way to go independently for each of them, and then we're doing it all together in one go and seeing how many convert. So let's take this third smile, or third 3 by 4 pixel image in. Kayla, what's our total?

[00:04:12]
We've got two negatives and a positive, so what's our total?
>> Kayla: One.
>> Will Sentance: So we've got two negatives, -1 by 1 is?
>> Kayla: -1.
>> Male Speaker 1: Plus 1 takes us back up to? [INAUDIBLE]
>> Will Sentance: 0, +1 1 by -1 takes us to?
>> Kayla: -1.
>> Will Sentance: -1, so our total is?

[00:04:30]
>> Male Speaker 1: -1.
>> Will Sentance: -1, does that equal -1, Kayla?
>> Kayla: Yep.
>> Will Sentance: You bet it does. Okay, so our first round of tweaks, otherwise known as iteration, doing of, repeated doing of, iteration of gradient descent. We'll see iteration of training, getting our multipliers to be better. Our first round is doing bloody well so far.

[00:04:58]
3 out of 4, accuracy, but shoot, that's where it ends. What's our total here? All 0s, all 0s, +1 by 1, yeah, the -1s are all 0s, aren't they? So we get one, well, anyone tell me what the output is?
>> Male Speaker 1: 3.
>> Will Sentance: Yeah, Maria, well, everyone said it, so.

[00:05:23]
1 by 1, Maria, is?
>> Maria: 1.
>> Will Sentance: Plus 0 by -1 is still?
>> Maria: 1.
>> Will Sentance: Plus 1 by 1 is?
>> Maria: 2
>> Will Sentance: Plus?
>> Maria: 0.
>> Will Sentance: 0, thank you, plus?
>> Maria: 1, so 3.
>> Will Sentance: 3 plus?
>> Maria: 0.
>> Will Sentance: Beautiful, 3. Perfect, is 3, Maria, 1.

[00:05:44]
>> Maria: No.
>> Will Sentance: No, it is not.