Machine Learning Demo

[00:00:00]
>> Let's actually do something fun. It's pose demo, what it's doing? It's simply gonna grab the image from the web camera. We can probably just, yeah, let's mount it a little bit. And it's analyzing the image and it's trying to figure out a couple of interesting points. For instance, those five points on my face, they correspond to you nose, eyes and the ears.

[00:00:25]
It's a little bit lagging, so if I'm moving too fast, it's just because it's updating the image first and then the overlay coming as the next frame on top of it. But on top of just tracking the face points, it can track the whole figure. I'm not sure if I can stand here, yeah, I will risk pulling the mic.

[00:00:54]
Yeah, you cannot see it exactly. But the thing is, the whole figure, all my hands, they can be tracked. [LAUGH] Okay [LAUGH] try it yourself. If you just go to the GitHub repo and click the link, just allow your web camera to be used. And do you want to try multiple people in the same frame?

[00:01:23]
[LAUGH] Try to wave your hand or something. [LAUGH]
>> [LAUGH]
>> It's pretty cool, all right, play around with it, it's pretty awesome. And it's using machine learning. So right now the architecture is MobileNetV1, right here, I can highlight it. You can actually switch from mobile to ResNet, but ResNet is significantly bigger.

[00:01:59]
So right now, we have six frames per second, right? You can see the image is a little bit sluggish. If we switch to ResNet I think it will drop to something like three frames per seconds. Maybe even less than that, zero, [LAUGH] no, back to three. Okay, but you can improve the speed if you for instance, change the input resolution.

[00:02:25]
It's not the resolution of the image shown here, it's the resolution of the image provided to the neural network. Yeah, you can see it, it didn't affect the performance of the model itself, but significantly improved the frames per second. And if you're gonna be playing around with it, right now algorithm is multipose.

[00:02:47]
It means it will be analyzing multiple people. You can switch to single pose, which will increase the performance. But unfortunately, it seems that Google guys [LAUGH] made a bug in the code. Because right now those points are mirrored. So you can see [LAUGH], the overlay at least, is mirrored.

[00:03:08]
So, yeah, we should probably file a bug report to them. But anyway, it's a really fun demo, and you can play with it yourself and just see how it works there. A couple of things you can do, just turning on and off things and kind of modifying the parameters of the model itself.

[00:03:28]
It's pretty fun, and the source code is available. So if you just go to TFJS examples and look for PoseNet. That's the demo with the source code, the one they used.