Defining the Solver & Loss Function

[00:00:00]
>> So we defined our model, right? We have this flatten to dense layers. What we need to do next is to tell, can define the solver. So solver will contain basically two things. We will need to specify how we want get to the right solution and what should be our loss function.

[00:00:23]
So if we say something like this model.compile, and that's basically completion of our solver for that particular model. And as the parameters, we can specify optimizer. So we can use something like stochastic gradient descent, SGD, or adam. In the previous example with linear regression, we used something similar to stochastic gradient descent.

[00:00:55]
We started with some random point and gradually through descending in our gradient space, we get to the extremor of the function or sorry, mathematic jargon. [LAUGH] Let's say the solution, the minimum of the function, in this case, it was the minimum of the error function. In our case, adam is just another solver which can get to this point a little bit quicker and we still need to provide the loss function.

[00:01:26]
So loss function, there are several variants what you can use here. By the way, just out of curiosity, if we type something like this, no. I think I need to specify a question mark after the function to read about it. So if I type model.compile and then puts question mark, yeah, I can read more about the function.

[00:01:53]
So for instance, the default optimizer is rmsprop, which stands for Root Mean Square Prop, I don't remember what prop stands for. [LAUGH] But it's just another way to get to this minimum. And for the loss function, yeah, there are different already predefined losses. So we can actually go directly to TensorFlow losses.

[00:02:21]
The beauty of Keras is that it can use TensorFlow functions easily. So you can actually mix TensorFlow and Keras and vice versa quite easily. So what we can do, we can simply type TensorFlow losses, and just look at available losses to us and there are plenty, right? So for our problem, just because we have ten neurons with Softmax, it's usually recommended to have sparse categorical cross entropy.

[00:03:00]
Let's see, do we? Yeah, cross entropy right here. What else do we need? So let's say optimizer, loss function and metrics. Yeah, we need to specify how we gonna measure our success. So let me just close this one, metrics. And once again I can use TensorFlow predefined class, or you can say I will just measure accuracy.

[00:03:31]
Oops, I think I misspelled something. Let's see, what did I misspell, accuracy, okay. Probably it should be capitalized. No, I know what the problem is. The thing is, with the metrics, I'm providing the whole list of things I'm measuring my performance by and accuracy can be just one of them.

[00:04:00]
So let's run this code now, still a mistake. Expect it, and that was a mistake. Watch it return. Okay, one second, let me quickly take a look at the code again. So okay, we have, Let's switch to pure Keras encoding. So with optimizer adam with the loss function, we can say sparse, categorical_crossentropy.

[00:04:54]
I have a question. Yes. Is it possible that it could be ACC, ACC for accuracy? I think it was with the previous version of TensorFlow and they switched from ACC to accuracy kind of full spelling, but we can check it. Yeah. Okay. Accuracy? Did I misspell it? No.

[00:05:17]
Accuracy, okay, it worked. [LAUGH] I know what happened. I use TensorFlow but yeah, I should have basically specified to create the object of that class and that provided the class itself. Basically, if you spell this with words, right, that's the recommended method of referring to different optimizer and losses in Keras.

[00:05:49]
And I think for the beginners, it's the optimal way. If you want to switch to TensorFlow optimizers for instance, right, and then adam. For instance, if I execute it right now, it will also create the mistake, because I need to create subclass and now it's working. But it also allows you to, for instance, specify additional parameters.

[00:06:17]
So different optimizers, you can change those hyper parameters like beta and epsilon. It kinda defines the behavior, how our optimizer will be getting to the solution. But we can leave all the default values right now. It doesn't really matter.