TensorFlow Model Layers

[00:00:00]
>> We have the functionality to click on the button and start keeping some data, in memory or at least, not the old ones, but the latest ones in memory. And we have our function to get the image. So maybe below we have a training button that we declared at the beginning here to start the training.

[00:00:19]
So we're going to add a onClick event so train button. Once we're ready to train, we have an async function. And we are going to call the function we haven't called yet. Basically, we're gonna call it train. And what we want is to maybe update the UI as well to say that now we're kind of in training mode.

[00:00:42]
So the status element, we were hiding it before when the model was loaded, so now we can just bring it back. But just change, its innerHTML. So we have statusElement, innerHTML, and just gonna set it to training. So, we have a function that we need to declare, but at least the UI is updating to tell the users like, wait a minute, I'm training, basically.

[00:01:10]
Okay, so now let's create our train function. And this is where we're gonna declare a new model. So const train and then I'm opening function here. And because we're just starting the train function, what I want is to set my isTraining variable to true this time. And before I move on, very important before we crash everyone's browser is that remember a little bit earlier when I'm setting mouse down here when I'm adding the example.

[00:01:48]
Let's make sure that when we have an on mouse up event as well, we're actually setting that mouse down to false. Otherwise we'll be stuck in a loop for long time. So if you find, again, that block or you can write it anywhere. But I just like to keep it together.

[00:02:02]
So here in the buttons container, we have an unmatched up. So once we stop actually pressing that button, we're just want to set that variable back to false. So that means that once we stop clicking on the button, the mouse down will not be false. And we will not be stuck forever into this loop.

[00:02:24]
I've done that before and it crashed my browser. So that's what I'm telling you. We can now go back to our training function, okay? So here we're setting isTraining to true and maybe the first thing we wanna do. I like doing that it's not mandatory. But usually I say, okay, if, oops, not that.

[00:02:50]
If there is no examples that we already added, I just want to either console.error or anything. And I want to tell the user you forgot to add examples before training. Because if you call this, and actually you should probably throw this error cuz we want to stop the rest of the logic.

[00:03:17]
It's not gonna do anything if it's gonna try to create a model and training on data that does not exist. So if the user did not yet created or added some examples in the function that we created here, we should tell the user, stop. And yeah, and then hopefully the user will add some examples and then we'll be able to run the training.

[00:03:42]
So okay, we are slowly getting there. Okay, so to be able to train something, we need something to train. So we need a model. So at the beginning we created a variable called newModels. So now we're gonna actually do something with it, and this is where you're going to see the syntax on how to actually create a TensorFlow.js model.

[00:04:07]
So we have our new model variable, and I was talking a little bit earlier about the two types of models, like sequential and functional. So here we're going to create a sequential model, and we're gonna add multiple layers. So the way you do this is tf.sequential. And this going to take an object here, and the parameter that we're gonna pass it is layers.

[00:04:30]
So we have a layer, and that's going to be an array of TensorFlow.js layers. So first of all, we are going to pass it something related to the initial model that we had at first. So we have tf.layers. It's like how you start to create layers. And the way that the did it.

[00:04:56]
The first layer is flattened and the inputShape is going be what is expected by MobileNet. Because again, we're building on top of that model. So we have the initial model, oops, initialModel. And the output of this model's outputs, and then we are shape and slice. So that will return to you the input shape that is expected by the next layer of the model.

[00:05:27]
So it is a bit complicated. It's not really something that you can know at first. It's just, to me, I just know that that worked for my application. But there are times when you might have to try different methods on the object and seeing. So we have, in the very first layer, we kinda have an input shape of what's expected from MobileNet.

[00:05:52]
And then we can create another layer that starts to be a bit more custom. So we can do tf.layers and that's gonna be a dense layer. And that's the ones I've mostly worked with as well. But there's different parameters that you can start to set into this layer.

[00:06:14]
And it's kind of related to what I was talking when I introduced this section about activation and things like that. So first of all, we have units, which is the dense units here, the variable that we declared at the top that I think was set to 100. Let me scroll back.

[00:06:30]
Yes, so it's like we're kind of expecting that the output of the first layer is going to be 100 units. And they're gonna be fed into the second layer. And here we have an activation function that is going to be set to relu and I forgot what relu means let me maybe check quickly can I.

[00:06:57]
I know that sometimes it's a rectified linear unit. So sometimes because it's all about data and algorithms and stuff. Sometimes you might not know exactly what they do, but you might know the ones that you want to use depending on your application. So it's been a trial and error thing as well for this.

[00:07:19]
But for now let's use that one. Again, then we have a next parameter that's called kernelInitializer, and this is also, I think it's varianceScaling. And these ones as well, sometimes I do not know exactly why they're better. But I know that in the context of doing transfer learning this has worked for me.

[00:07:45]
I think tomorrow when you create our own model and we have different parameters, we can try and see what works better. But for now, we're gonna use that one and useBias is a habit set to true. Let me check. Okay, bias vector. I'm not sure that's necessarily completely needed.

[00:08:12]
Let's leave it to true. Use, sorry, not us. Use and it could be fun to actually see we're gonna make this thing work. And then we can try and tweak some parameters and see if maybe when we set it to false, it changes the output or things like this, okay?

[00:08:29]
So we need one more layer because at this point, we need the units. You want your last layer to have a number of units that's equals to the number of labels. So here, 100 is too much because our labels are only two. So we're gonna just create one last layer.

[00:08:44]
You could have a lot more, if you if you want, as well, tf.layers. But in this case here, we're not gonna do too much, we're just gonna have three. And as I said, the units here is supposed to be the number of output based on the labels, at least for the last layer.

[00:09:02]
So we have a labels, so it should be two. And then in terms of kernel initializer, we're just gonna use the same. Let's keep it simple. Maybe we can also use bias as the same value as well. And in terms of activation, for example, for that particular layer, I use softmax, but it would be fun to try it, okay?

[00:09:23]
Does using actually don't know yeah. So we can try with this and then tweak and that's what is usually called kind of parameter tuning is that you try different things and see what works better for your model