Optimizing, Batching, & Training

[00:00:00]
>> Okay, so I know that it's probably confusing if you're doing the first time. Even me as someone who's done it a few times, sometimes I forget exactly what, for example, Kernelizer does, but we can try to see if it works and then change the values if we have an issue with that.

[00:00:17]
But at this point, we created, technically, we kind of created the structure of our sequential model that is stored in our new model variable. But it's not yet created, we just created the structure, but we have not given our training data to the model, so it's not really available yet to use in the browser.

[00:00:34]
So, then we are going, there's like two more steps before we fit the data, is we are going to create an optimizer, so const optimizer, and this is going to be an algorithm that's called, so tf.train.adam, and we're passing the learning rate. And as I was talking about previously, the learning rate is how often the model is updated with when the weights are changed.

[00:01:07]
And this again, that's a type of optimizer that uses the Adam algorithm, but you can see here in the optimizer function of the docs that there are more. So if we want, we could change it to either grad or momentum. They don't really tell you which one is best, because then we would have to go into a deeper thing of, what's a momentum gradient descent?

[00:01:33]
And for what we're building, it's not necessary to know right now, what this does, it's more that you can try different ones, see the output. And then if you want to get deeper into maybe be more performant or try different types of output and see what happens, then you could dive into the docs and get a lot more information.

[00:01:51]
But here, once we've created our optimizer, we need to call a function called compile. So before we are able to train the model, we need to call the compile method and we need to pass it that optimizer. So, I think we might be able to shorten it but just in case, gonna pass the optimizer, and we have a loss function that is called, okay, categoricalCrossentropy.

[00:02:24]
And here in terms of losses, you want your loss to be as low as possible by the end of the training, because it means that the accuracy is better. If you end up with a loss parameter that's too high, it means that it's not accurate enough. So here in terms of the loss that we're setting here, categoricalCrossentropy might be actually a function to calculate the loss that you get or a type of algorithm.

[00:02:59]
Actually, let me Or the type of matrix that you get. Okay, so I think that whenever you're getting the loss is calculated in different ways. And here I'm using categoricalCrossentropy, but I feel like we might be able to use different ones as well, but we can tinker with that later.

[00:03:21]
So when you create a model, just a quick recap, you choose your type of algorithm, like sequential or else, and then you add some layers in it. Then you optimize it when you compile it with an optimizer, and the next step will be able to actually train that model with data, but first we're going to split our data by batches.

[00:03:46]
So the first thing that we do, have the batchSize variable, and we are going to use our bySize fraction variable that we use before as well. But first we do Math.Flaoa, to have a round number, and if you remember, our example data is stored in a variable called xs, and you can call the shape function on it as well.

[00:04:09]
So here we're going to kind of get the first input of our examples, and we're gonna multiply by the batch size fraction, and that's going to give us the batch size based on our examples. And here we could have like an if statement that says if the batch size is zero, then, kind of like stop the logic there.

[00:04:35]
But for our example, well, let's forget about error handling just for a second, and we should be fine because we are going to add examples and things like that. So at this point, when we've have created the structure of our model and we've compiled it and we have a batch size, we can call finally, the function that's called fit.

[00:04:54]
And fit is going to actually be training the model. And what you pass into fit is that you pass your data, so your examples xs, and you also pass your, sorry, actually xs, and what did I have? I think it was xy. Xy, so that should be it.

[00:05:11]
So we're giving it the data, we're giving it the labels, and then we need to pass it some parameters, like our batch size. So, because at first, xs contains like everything, it's not split in two batches. So first we're passing it our batch size that we declared just above, and then we also need to tell the model how many steps are you gonna go through to be trained?

[00:05:32]
So we need to pass the epochs, that I defined at the beginning as well. So the number of steps, I think we probably should be able to just do that, cuz we defined it at the beginning. And at the end of each epoch, so at the end of each process, we can have access to a callback that kind of can give us an output every time that the model has finished one epoch.

[00:05:57]
So it's kind of called callback. It's weird that they're not calling it callback, but it's called callbacks. And we can have a kind of sub event of onBatch end. So every time that you've finished or that the model has finished kind of like doing things with one of the batches, we can have an async function where we're passing it like the batch.

[00:06:16]
And we get some logs out of it, and we can then update our status element on the page so that you can see where we're at, at least in the loss function, for example. So we have status element, we are going to update the innerHTML element, and we can have a message like, so what I had worked with is, you are telling the user that the loss is we're gonna take the logs.

[00:06:45]
And we're gonna extract of it, the loss, and we're just gonna be, I think toFixed5 because I think it's a flow number and there's a lot of our use in there. So what this is going to do is that it's going to train the model with our data, with our label.

[00:07:01]
It's gonna split it into batches. It's gonna run the number of epochs that there is. And every time that an epoch is done on each batch, it's going to tell us the loss result of that training step. So then we'll see that it will hopefully decrease as the model gets more performance, and in the end, when it kinda stops deleting, we'll know that the training is finished.

[00:07:24]
And because the training should be finished after this, we should be able to set R is training to false. So as I was saying, it is more complicated than the things that are magic, but for now we're going to take a bit of a break and after that we'll be able to actually run the actual prediction.

[00:07:45]
Cuz now our model should be ready to almost being used, we just haven't done the prediction part of it.