Augmented Reality

[00:00:00]
>> Now, looking around the camera, the next topic, that is a huge topic, so I'm just going to take the most simple part is Augmented Reality that also mixes with virtual reality. So, Augmented Reality, AR or XR. Today for the web, for web developers or for web app developers, there are many ways that we can use.

[00:00:25]
We can use WebXR, that's an API available today on Chromium-based browsers, but in Safari, it's actually an experiment. Let me prove that. An experiment that is still not working. When you walk through experimental features, at the bottom, you have WebXR, okay, a lot of WebXR experiments. Why is that?

[00:00:48]
I mean, it's not 100% confirmed, but we know that Apple is working on some kind of headset. So they're preparing Safari for when that's ready, right? That's kind of why it's coming. So with WebXR, it's a low-level API. I will show you how it works in a second.

[00:01:11]
But instead of doing that, something that is probably more straightforward Is to use a high-level solution for augmented reality content. I will show you one in a second. Talking about WebXR, the low-level API, actually how the API is designed, is pretty similar to the Gamepad API because it was designed at the same time.

[00:01:35]
In terms of how it looks like, I'm not saying it's the same, but how it looks like, how the terms used within the API, is pretty similar to the Gamepad API. You can query about the devices available. What's a device? It can be a headset, or it can be a headset where you put your phone in, that they're kind of dead today, okay?

[00:01:58]
Daydream, there were a lot of projects around that, but today no one is using those. But on computers, if you have a VR headset connected, then you can detect that you have a VR headset. You can also detect on compatible devices 3D space. So it can let you analyze your space around you and give you a 3D model in JavaScript.

[00:02:21]
The pose of where you're looking at within that virtual environment, depth sensing. Again, this relies on the hardware that you might have, or software, because I will show you that you can use this even without devices with the camera. And the API will also create vectors representing movements of input controls.

[00:02:43]
So if you have an input control where you were supposed to move into a virtual environment, it will give you the backdoor so you can apply that into your web GL or web GPU environment. But that's complicated, you need to get into VR. It's really complicated. And also something, that's the last thing they have added, lighting estimation.

[00:03:06]
So based on the real lighting, it can estimate the light that you need to apply to virtual objects. And hit testing is for example, if you're walking around and you have elements, if those elements will hit real tables, real chairs, and so on. So working with the WebXR, we're not going to get too deep with this.

[00:03:27]
I will show you something I think that is simpler for the first approach to this, but you just called navigator.xr.requestsession. And there are three kinds of sessions today available, it depends on the device. Immersive-vr, that means you have a headset for VR, immersive-ar, you have a headset that can also look what is your real environment or inline.

[00:03:54]
Inline can be used with your phone without any device attached. Just using the camera and sometimes some sensors from the camera, it's actually a hardware. So Apple has created a format that is called AR Quick Look, Augmented Reality Quick Look, I'm not sure if you have seen this before.

[00:04:19]
Where you create an anchor, so a link with the rel ar and you point to a usdz file, that is a 3D model with metadata for augmented reality such as how that element fits in the real world, such as the real size of that element, that 3D model doesn't count for the real world.

[00:04:44]
Well, this is a 3D model with more metadata. On Android and Chrome, instead of applying this non standard way of doing things, they have created something known as model viewer. Model viewer is a web component that you can add to your website. And internally, it will use the apple's implementation or WebXR to offer the same solution.

[00:05:16]
So it's a library, okay? But internally, it makes the same code compatible, the same model compatible with iOS and Android. Probably you don't understand what that means yet, so let me show you what this is. I have here the modelviewer.dev website, let me refresh. So you see the experience.

[00:05:38]
So the right experience is that I am seeing that arm chair, and I can view that in my space. Can you see that View in your space? So View in your space, see if though I'm in the Chrome, yeah, I'm in Chrome, because I already granted permission. I wanted to show you the permission for that.

[00:05:57]
Yeah, can you see that Permissions, Augmented Reality is already allowed. Let me reset it so you see it from scratch. So this is what you see, that this website, so it's a capability that lets me create a 3D map of my surroundings. And we'll say, Allow. And now, probably you've seen this on iPhone as well, where it's asking me to move around so it can actually put, this is our real environment here in front of the mouse parts, it looks nice.

[00:06:32]
It's actually pretty similar to the real one you have there. I can even rotate it, okay? And I feel is a real size of that particular thing, okay? Pretty cool. So well, this lesson, it's Augmented Reality, but you don't need any device, just a website pointing to a model.

[00:06:52]
Also it depends on the phone, it might have or not hit test. If you have hit test, you will know when you are hitting another element. Not in this one, for example, you can see it's hitting, but it's not really hitting that. But anyway, you can try the different examples here, go back.

[00:07:10]
There are more elements that you can try, and also you can see this. That's if you go to AR, I can put an astronaut here as well. He can continue the workshop or maybe he is a she, I don't know. There you have it. This option can only be viewed in true scale, it's too big.

[00:07:36]
Anyway, for that, that's a capability and it's available on iOS and Android as well. To make it compatible with both platforms, the simplest way to do that is to use this library. Internally, it will use WebXR ON Android or the Safari solution for iPhone. And you can reuse the same model, the same file.

[00:07:58]
These things are really nice for something really quick. But having your phone like this, like with this pose for a while, it's a little bit annoying. So it works, it's nice, but no more than one minute doing this. So you don't wanna do that. For that, you wanna a headset.

[00:08:17]
So if you have a headset, yeah, that's much better because you don't need to have your hand like this or with an iPad with two hands doing this all the time. You will get tired pretty easily. So it looks nice, I mean, it's a geeky thing, but then when you try it, it's just for a couple of seconds and that's all.

[00:08:40]
A headset will change the landscape a lot. Here you have another example I did a couple of months ago. I don't know which country, it was a hotel, that's the only thing I remember. But it works pretty well. As you can see, it looks really well. That was iPhone I think, and that's an animation, so everything can be moving, as you can see.

[00:09:07]
So it can be an animation on the fly that you receive from the network that is created by another user. So technically, everything is there.