Sensors API

[00:00:00]
>> This is the new API, remember, not available on iPhone, and iPad. We need to use a constructor, a constructor differs per sensor. For example, in this case, we can read the magnetometer, we can select the frequency, the frequency is in seconds, and then, you listen for the reading event over that object.

[00:00:23]
And in this case, we are getting three values for the magnetometer. And then you start reading, you can start reading, you can stop reading. Make sense? The old API, you just read the values and you can't stop reading the values because for the old API, your API was only supporting magnetometer, a gyroscope, and accelerometer.

[00:00:47]
And all those three sensors are typically always on. Actually, most of the sensors are always on because those sensors are being used, for example, to change the orientation of your phone. When you are in one specific app and you go landscape, it uses those sensors. So the sensors are on, so using those sensors don't have a cost in battery, for example.

[00:01:13]
Because they're always on anyway, you're just reading the value for your own app. So these are the sensors available and the constructors available today on this generic sensor API, the one that is not working on iOS and that's a permission policy in case you wanna enable or disable that from the HTTP header.

[00:01:37]
So you have an absolute orientation sensor, linear acceleration sensor, gyroscope, ambient light sensor, magnetometer accelerometer Okay, make sense? And do we have any Android user here in the room? Okay, four. Well, because on Android, we can use this API on Chrome and Android. So that's my Android, okay?

[00:02:01]
So you you can see I have my Android here and you're seeing a mirror of the screen. So on this is firtman.github.io/capabilities, the URL for these workshops. And they're "Cool demos to try", okay? So if you have an iPhone, those demos, the first demos won't work because they're using the generic sensor API.

[00:02:23]
So for example, orientation phone. In this case, you can see I'm moving my phone and it's applying CSS 3D transformations based on that. Is that useful? Again, it depends. It's a nice effect for a website. By the way, did you see any permission dialog? No. So on this API, on Android, we don't get any permission dialog, you just read this.

[00:02:53]
So you can apply CSS effects to your websites right away, okay? For your titles, you can do a little game, a gravity game, or things like that without any issue. Then I can try the 360 degree. This is cool as well, so I can move my phone. And this is a built on window into another, it's like a portal into another dimension, okay?

[00:03:20]
Like, we are in the beach right now, okay? So, and you can see it works pretty good. Every axis, okay? Those are open source by the way, so if you wanna check the code and use them, you can. These demos are coming from Intel. And also you can get the raw data.

[00:03:42]
So I have here a sensor info where you can actually add different observers like sensor observers, so I can add, for example, which sensor do you want? Let's try accelerometer. The frequency, I can use 60 and then I can turn it on. And now you can see the raw values that the phone is giving that API.

[00:04:09]
So you can even check, okay, where are the values? You can see that you won't get integer values. So typically, when you're using the sensor APIs, you don't need to check absolute values, just check ranges. Between -0.1 and 0.1, don't ask for zero because it's difficult to get really zero.

[00:04:31]
So you can see, by the way, why it's nine dot something close to 10? Can you.
>> 9.8 meters per second squared.
>> 9.8, that's gravity, Earth gravity. Of course, we can talk with Elon Musk and try these on Mars, okay? And see, yeah, it will change, right?

[00:04:51]
So we would see a different value here in JavaScript on Mars. Anyway, of course he if I turn the phone down, you will see -10, okay? On different axes, it depends on the axis. So that's the accelerometer and you can try all the other as well. I don't wanna try all of them but I know the gyroscope, okay?

[00:05:16]
And now you can see the bias and gyroscope based also on what you're doing. Let's add another one such as ambient light. So, what you see there is that it says error. So what's going on with ambient light? Ambient light also used to have an old API, okay?

[00:05:41]
An old API. So classic API not using the generic sensor API. In fact, if we go to "caniuse", by the way this is the first time we're going to "caniuse", this is a website where you can check if you can use one particular API. So I can check, for example, sensors or sensor, orientation sensor and we have ambient light sensor, you can see it's pretty red.

[00:06:06]
There are some greens there and then there is a flag, let me increase this a little bit. So what's going on here? It's under a flag on Chrome or Firefox. It was green between some versions, but then they removed that. Why? Because they realized that it was triggering some privacy issues.

[00:06:33]
It's weird, that's why we can't have nine things. So, someone realized that when you have a tab, okay? And let's say you open an app that is using this API to read the current light on your room, and then you open another tab and you open Tiktok or Tinder, okay?

[00:06:54]
Someone realized that the brightness of your screen is also affecting the light around the sensor. And someone could actually get what are you seeing in other tabs or in which websites were you, just by reading, light changes, okay? Makes sense? So because of that they say, we have a privacy issue here, so let's remove it and it's still there under a flag.

[00:07:24]
So we can still work with it, okay? If we enable a flag, it's experimental. I think on Chrome, for example, I think the flag is for generic sensor extra classes, but you can still go and enable generic sensor extra classes. So for example, I can go to Chrome here, go to Chrome or Edge, flags and enable extra search for sensor generic sensor extra classes, enable.

[00:07:57]
So this will add ambient light sensor and magnetometer on Mac, Windows. I can relaunch this. So now I can try this on my desktop computer, but also I can enable that here as well. So I can go to Chrome, flags on my Android, search for the same generic sensor extra classes, enable that, relaunch Chrome and now it should work.

[00:08:29]
We can try, I can go back and use my real phone for that, let's see. Oops, add sensor, ambient light, frequency, 60, add. Okay, so now it's getting me 0 for some reason, and only 0, so it's not changing. But it's an experimental, let's try it on desktop to see if that works.

[00:09:03]
Do I have it here? I do. We are talking about the sensor info, that's the link. So we're going to try, ambient light 60. Let's see, let's activate the sensor. No, not readable. Why is not readable? Probably my Mac is not sending the information to Chrome, that's one reason.

[00:09:26]
You can try it on your Mac, depends on the Mac, I'm on an M1. So it depends on that. You can read that, you can try it in Firefox, but anyway, you know that that sensor is there but it's still really experimental. So that's actually the yellow side.

[00:09:41]
In this case, it's not the API, it's just that one, and by the way do you have accelerometer on a computer on a desktop computer and a laptop?
>> I mean, I know that the old ones,
>> The old ones have, which ones the ones with just with a hard drive, not with an SSD.

[00:10:02]
Because the hard drive on the Macs, they used to have an accelerometer to know if your computer is dropped, so they can lock or actually, yeah, lock the hard drive. Anyway, so today, those sensors are more optimized for tablets and phones, okay? Makes sense? And the ambient light, yeah, we're not there yet, okay, to use.

[00:10:29]
Okay, what about the iPhone? So as I mentioned, that API is not available on the iPhone, but you still can read accelerometer, gyroscope, and magnetometer, how? Just adding an event listener to the window object, so global events, just that. Add event listener, device motion, device orientation. And you will get the same values as with the other API.

[00:10:55]
So you can do the same effects, okay? The same effect with the phone that moves. Or even you can do the same virtual door portal to another dimension, it's possible in iOS. There is a however, since iOS 13, that's a few years ago, Apple realized that they also have privacy issue with this.

[00:11:26]
There were ads, iframes that were reading your sensors to try to fingerprint you, okay? Because of that, they created a permission dialogue that they didn't document anywhere. You can go and search for that, you will find my blog post on that. So and I got it looking at the WebKit source code actually.

[00:11:57]
And then someone from the WebKit team confirmed that on Twitter. So at that point, they were not documenting anything. So actually, if you wanna use this on iOS, you need to request permission and the user will see a dialog, okay? So then no one will do that in an ad, it doesn't make any sense.

[00:12:17]
So if you have a game or an immersive experience on iOS, you have to request permission on device motion event. And if it's granted, it will give you a response as a string, granted or denied. If it's granted, then you can start listening for these events. If not, it's not going to call your events.

[00:12:37]
In fact, there were a lot of websites and games using this. And from one version of iOS to the other one, everything has stopped working and no one knew why. It's because Apple added that permission dialog as a requirement to use the API, okay? Make sense? So, again, the sensors that you wanna use are accelerometer, gyroscope, and magnetometer, the rest are very experimental.