Picture In Picture Possibilities

[00:00:00]
>> And with video, you can also use picture-in-picture. From JavaScript, it's actually pretty simple. You take the video element and you'll requestPictureInPicture. And if you wanna get out of picture-in-picture, you talk to the document, not to the video element, and you exitPictureInPicture. And there is a boolean in the document element, pictureInPictureElement that tells you if there is an element in this dome that is currently in picture-in-picture.

[00:00:34]
So, for example, if you have a button that is toggling picture-in-picture, on and off, that's the code. And I will show you this in action in a second. Any question, yeah? Okay, so also there are new events for the video element like enterpictureinpicture, leavepictureinpicture, okay? And as I mentioned before, you cannot add HTML.

[00:01:00]
So if you wanna customize the UI of that secondary canvas that you have in the background, you have to play with the video that you're sending, the video buffer that you're sending, okay? To the API. There is a pseudo class for CSS in case you wanna customize a little bit, the video element, but there are not so much things you can customize, actually.

[00:01:28]
And, well, we have already seen the JavaScript code that will let you toggle this. And controls are managed through the other APIs, because the picture-in-picture can have controls. And those controls will trigger JavaScript through the other API, okay, make sense? Cool, so before actually seeing this in action with our code, I wanna show you this, pip-rendering-fun, picture-in-picture rendering fun.

[00:01:57]
So let me run this, for example, on Chrome. Remember I mentioned that there are some tricks with picture-in-picture. Well, here you have a couple. For example, this demo will open picture-in-picture, not with a video, but with some stats. Let me show you this. Of course, it's a picture-in-picture so you can move it around the OS.

[00:02:21]
And, of course, I can minimize the browser, and that's picture-in-picture. And also you can even mix this with actions. So, for example, I could add a Previous and Next button over the picture-in-picture that will trigger JavaScript, and that JavaScript will change what we're rendering. This trick doesn't work on mobile, only on desktops.

[00:02:48]
So mobile, you have to send an actual source that is actually a video. If not, it's not taking it, okay? [COUGH] So there are a couple of examples here. Meta is just showing the window size. So if I resize it, remember, it feels simple, but this is actually a video stream, okay?

[00:03:15]
On top of the web app. Well, time tracker, there are many, okay? As you can see here, of course, you can resize this window, you can see it can have it's any size with Any orientation. So it can be landscape or portrait, it doesn't matter, it just work.

[00:03:41]
Let me go back to the window. And this one is drawing so you can actually draw in a canvas, and it renders directly in the picture-in-picture. This is an example, again, yeah, a little over the edge. But to show you that it's another way that we have, at least on dekstop, okay?

[00:04:08]
To create the UI that runs in the background and keeps a UI floating over the OS. Yeah, also even if you don't provide any JavaScript code on compatible browsers, any video element will have an icon. In fact, in our code, I didn't add any code here, I have a video.

[00:04:30]
And if you start playing the video, you could start picture-in-picture even if I'm not adding any JavaScript code. So picture-in-picture will work anyway, no matter if you are using the API. The idea with API is that you can actually customize what you're doing, you can listen for events, and you can request that from code.

[00:04:53]
But now if I minimize this, that continues playing.
>> How are you dynamically generating a video stream to feed into that? Is that-
>> Well, this is open-source so you can actually see it in action, okay? This one, here I have the repository. But it has to do with creating a canvas is rendering on a in-memory canvas, 2D canvas.

[00:05:17]
And each image from the canvas, it's sent as a buffer, like a bit map, as one frame of a video.
>> Can do that? There's-
>> You can do that. Yeah, you can do that with the web.
>> Bonkers.
>> People put in the background for games. Render a frame before they need to render it.

[00:05:35]
>> In fact, think about this. You have the gamepad API, because maybe you're thinking, okay, I do this in the background. Might have a secondary nice picture-in-picture, but how can I play with the game? Well, you have the actions, but it's actually pretty limited. But now you have the gamepad API compatible with every browser.

[00:05:56]
So you have a gamepad, a bluetooth gamepad, or wired gamepad, you can actually use a gamepad to play a game in a picture-in-picture background web app. So it's possible, but to actually make that work, you need to request picture-in-picture, your code or the user. So someone needs to request picture-in-picture.

[00:06:19]
If you don't request picture-in-picture when you go to the background, you will lose a lot of abilities. So it's not gonna work properly. But if you request picture-in-picture, your code has full background abilities as if it's in the foreground.