Detecting Parts of the Face

[00:00:00]
>> So here, what we're gonna do next as a first step is we're gonna use the values that are in this box object here. So as this is not TensorFlow specific code, I wrote it in the utils, but we can have a look at it as well, cuz it's basically using Canvas.

[00:00:17]
So if you've worked with Canvas in frontend before. We're creating a Canvas element and we know the image size of the image that we're taking when we're clicking the capture button. And yes, here is just styling a position date in a weird position, but you can change that.

[00:00:38]
And here what we're doing is that we're going to pass the image that we're taking and also faces is going to be the coordinates that I just showed you in the console. And here, if you are used to you working with Canvas, you might be used to it, but just in case here, you always call like beginPath before you draw something on the Canvas.

[00:00:57]
And strokeStyle is going to be our line and it's just the color. So we are gonna draw a red box around a face, and the rectangle is going to be using the values that were in the faces object. So, because there's only one face in the image or only have one face that was returned in the prediction, we're going to use the first one.

[00:01:16]
But you could try it yourself with two people and then you'll have to maybe duplicate this and use a face 1 instead of face 0. And then we have xMin and yMin for the top left corner. Yes, and then width and height. So if we look back, if I didn't refresh, yes.

[00:01:33]
Here, you have xMax and yMax. I think that will probably be the coordinate of the bottom right point. But if you already have width and height, you might not actually need these two as well. And then we're just appending that onto the image. So if we go back to part three, and we just import drawFaceBox.

[00:01:54]
And here, where we have our console.log(faces), we just have FaceBox and we give it faces, so the prediction from the model. What should happen if I kind of rerun the thing, so I turn on the webcam and I capture the face and then I predict, [LAUGH] I forgot.

[00:02:17]
So if we look back to the details function, the first argument is not faces is the image itself. So here we pass it photo, the same way that we passed it in our estimate faces method as well. And now we should work, I understand that you could not find min X and stuff in photos because it's not that wasn't the correct object.

[00:02:36]
So start again [LAUGH] and predict. [SOUND] You have a [LAUGH] square around your face. In this particular example, that's just what it does. But you could see how maybe if you were maybe not using, well, if you wanted to like track people, that sounds not really good. But if you wanted maybe to work with like a cars or something, if you wanted to see how many cars go past you like every day or something like that.

[00:03:04]
Sometimes if you look at examples, they do have these little rectangles to kind of isolate the object and to know where they are. But you could, let's say if you were building even an art installation or something and you wanna know how many people pass by everyday, then you could kind of like draw things like that.

[00:03:26]
So basically, to be able to see your face on an image and then draw something in the Canvas, then that's basically the lines of code that you need. So hopefully you can see how you can start with a small idea of using an image detection model with a file picker, and then you can experiment with either changing the model or changing the type of interaction.

[00:03:51]
And you can explore that type of things and see what's available with the TensorFlow.js library. And then you can even come up with different applications that you want to