Transcript from the "Shape & Face Detection" Lesson
>> Okay, so we've seen then how to use the Web Speech API for both recognizing what the user is saying and to let the web app speak. So now it's time to talk a little bit about the camera and visual. And we're going to talk about shape detection.
[00:00:17] Shape Detection API, it's actually today, let's say, between yellow and red. One of the parts is light green, it's Chromium-based only. The idea is to actually detect things with the camera without actually opening the camera and analyzing the image by ourselves. So for example, we have a BarcodeDetector.
[00:00:42] It works with 2D barcodes or QR codes, Chrome on Android only, if you wanna try this. It's actually pretty simple, three lines, and you have a QR code reader. It's not just barcode, we have FaceDetector. That's the API that I was mentioning that you have today. That will give you something, in this case, the object that you receive.
[00:01:07] Will give you coordinates within the image of the face as a rectangle. And each part, the nose, the mouth, and each eye. And you will get x and y, width, and height for where the eye is. That's all, okay, that's the face detector. And also we have a TextDetector that is basically OCR.
[00:01:29] So it takes text that you have in an image, and it will give you the text representation as a string. It's actually pretty simple to use, the problem is that it's not on iOS or desktop, it's just on Android. And how do we know which part is available or not?
[00:01:48] Well, the API is called Shape Detection. So let's, for example, go to Chrome status, And look for the feature, Shape. And here we have the Face Detection API, the Text Detection API, and then Barcode. If you go to Barcode, it says, enable by default. So that is shipped, so it's ready to use.
[00:02:16] Text Detection, in developer trial, so we need to enable the flag. And Face Detection also in developer trial. So in this case, if you wanna use them, it's pretty cool. Cuz I use that for create a quick web app for a kid. It's actually a kid that is not reading yet that can point to a text, and using a speech synthesis, it can read aloud what's in there, it's whoa.
[00:03:02] Or take an image and do the analysis for a QR code reading, face detection, or OCR. But it will take more CPU, and you will need to use a library, okay? So we're here have an example, that the one that reads the text, think. Look at that PAN AM, so hopefully, That's Google Flights a few decades ago, [LAUGH] and it will give you the text immediately, almost, on Android.
[00:03:34] So Media Devices got to do with the camera. I'm not going to spend too much time talking about these, because this has been for a while. There are a lot of content around how to open the camera. But Media Devices was known as a good use of media for a long time, because that was the name of the function, okay?
[00:04:13] And that's all, the API ends there. The thing is, what do you wanna do with the image? If you want to take a picture, if you wanna analyze something. Something that is kind of new. It's another API that is Chromium only. It's called Advanced Control Camera, that is also known as PTZ, pan, tilt, and zoom.
[00:04:35] That will let you manipulate pan, tilt, and zoom for a camera. So then you can adjust the input stream, you can have some controls over your page, adjust that. For example Zoom is using that, or these web apps that let you stream over Twitch, YouTube, Facebook Live, or Instagram.
[00:04:58] It's useful for mixing it with UI controls, where the user can actually zoom in. It's difficult to use this from your own code, for you, as developers, to decide which value to use. Typically, the user will pick the right value.