Cooking Masters Project Overview

[00:00:00]
>> Yeah, you can use this for gaming, but also, we're going to use it for something different. So let's get out of here for a second and let's load our project. Remember, our project is available in the repository, that is, github.com, my last name, Firtman/capabilities. So if you clone this, there is a CookingMasters folder.

[00:00:26]
So what I'm going to do is to open a local server on that folder, we can also try it online, but I wanna show you a local because maybe I wanna make some changes on the fly while we are discussing this, maybe we change it. So I'm going to open Visual Studio Code now on CookingMasters folder, I will take that as my root.

[00:00:50]
CookingMasters, there we are, it's a Vanilla JS app, so no library, no framework, no npm, no build process, nothing. So we have an index.html, it's pretty simple, header, we have a main with a couple of sections, there are a couple of pages, you will see it in action in a minute, but nothing really complicated, just HTML.

[00:01:19]
Then we have CSS, you know CSS, so nothing so important there. I have data. On data, I have recipes because this is a cooking app. The idea is an app that will help us while we are cooking, okay? So we are cooking food at night, at home, and I want an app to help me.

[00:01:40]
The idea is that because when we cook we are using our hands, our hands will get dirty, I don't wanna use my keyboard or my mouth or my trackpad at the same time. So we are going to use capability APIs to manage that app while we are cooking.

[00:02:00]
So we can use an iPad, an Android tablet, or a computer, put it aside and interact with our app touchless, that's one of the goals. Yeah, maybe if I'm using a gamepad, I'm going to attach the gamepad, yeah, but we will use other techniques as well. So let's run it first so you will understand what this is.

[00:02:22]
So I'm going to open a web server. This is how it looks like. So this is the homepage, Cooking Masters. We have a couple of suggestions. We have Argentinian asado as well, that's a barbecue, okay? This is the Argentinian meat thing, but anyway, we can take fish tacos.

[00:02:42]
So if I click there, I get into the details. This is a client-side routing system, pretty simple, with Vanilla JS. I have a list of ingredients, and here comes the tricky part, not tricky, but the important part, we can start cooking. And that enters a new mode. Here I'm using different capabilities, even some that I haven't mentioned yet, but we will.

[00:03:04]
If I click there, and you can try this on your computer, you can open this on Chrome, Safari, or Firefox, it will work, if I hit Start cooking, it goes full screen, okay? So it went full screen. And now I have, in this case, the steps. This is step one, I need to pickle my onions, okay?

[00:03:25]
And of course, I can go to the next step. There are some steps where they have some timers. For example, in this case, it says that, yeah, I need to let sit for at least 30 minutes, and there is a button there that says, hey, do you wanna add a timer for 30 minutes?

[00:03:46]
If I do that, I can click here and it adds a timer. Right now it's only for five seconds, always, just for us to test this, but we can change that, okay? This is hard-coded just because we are going to use that timer for something, and I can add more than one timer and for different parts of the cooking process, and then on the screen, I will have those timers.

[00:04:08]
If we click the timer, they've been removed, okay? So that's the app, okay? So of course, then I finish, and that's all, that's kind of the app. So I have a little button there, top right, that's a permissions section where you will see the current state of the permissions that we will use, okay?

[00:04:35]
Make sense? I will show you the code in a minute, this is using the permissions API for when it's possible, because remember that permissions API is not available for every capability. And for example, for the gamepad API, the gamepad API is not actually in the permissions API. But my gamepad right now is off.

[00:04:59]
I can turn it on through Bluetooth or through USB, I have both options. Both will work, in our case. It's not the problem, but I can show you that it works. Sorry, it works with USB as well as with Bluetooth. So if I connect this like so, At one point it's going to be connected.

[00:05:24]
So let me put it there. It's connected because it has a light. So what's the idea? The idea, for example, is that when I am cooking, also, I have a section here that is really grayed out with a couple of icons. One of the icons is actually an emoji, it's a gamepad.

[00:05:47]
So the idea is that I can click there, and now it says it's connected. So when it becomes alive, it means connected. Which means I can now use this to move things, okay? So I'm moving now my steps. And also with the button, I can get out, okay?

[00:06:12]
Just one thing, connector that I have. So now, if I go here, you can see that it says gamepad is connected, okay? So just for you to know when you use a gamepad, the gamepad, after a while, goes through some kind of a sleep mode. So even if it's still the light sensor this connector, so what you need to do, so if I restart, now maybe it's disconnected.

[00:06:34]
The way to connect that is to move something, to touch something, and it connects again, okay? So you just connect it again. By the way, in JavaScript, do I need to pick the gamepad as a user? Do you see any permission dialog? We didn't see any permission dialog.

[00:06:52]
The gamepad is one of the harmless capabilities, we won't see any permission dialog. If the gamepad is connected to the computer through Bluetooth or through USB, by the way, it works also on mobile devices, that's okay, you can use it, okay? It's like a keyboard, you don't get permission to use the keyboard, okay, on a website.

[00:07:16]
So the same happens with gamepad. But now let's try to look at the code because the gamepad API, it's not so simple as I was saying before. The gamepad API requires some little tricks, so if you go to scripts, let's go first to understand this web app.

[00:07:36]
On app.js, we have a DOM content loader where everything starts. We are starting the router, the router is the one that is managing different routes, navigation between pages. Then we have the cooking section that is available in Cooking.js, that's the one that is managing the steps going, next step, back step, the timers, everything is there in cooking, okay?

[00:08:03]
It's just coding, okay? You will see the code, this is Vanilla JS, nothing really complicated. And then we have a Capabilities JavaScript file. On the Capabilities JavaScript file, what we have here is the connection to every device or hardware that we will use during the rest of the workshop.

[00:08:25]
There is a full section here on, let me look for it, Gamepad, this is the gamepad section. So when we are adding an event listener for gamepad connected and gamepad disconnected, what we are doing is just sent him to the console, the gamepad is connected or disconnected, and calling Capabilities.load.

[00:08:52]
Capabilities.load, what it's doing is marking on the screen if it's connected or disconnected every API. This is kind of a manual work, because as I mentioned before, the permissions API is not available for every feature. So for every feature, we have a different code, and here we have, the code for the gamepad, okay?

[00:09:15]
Let's try to understand because it looks scary but it's not so complicated at the end. First, we get all the gamepads with navigator.getGamepads. In fact, let me do that in action directly here. Let's open, I was using Canary. Here I have Cooking Masters. Let's open the console. And let's say, I'm going to copy that line, gamepads = await navigator.getGamepads.

[00:09:51]
If I look at gamepads variable now, it says, it's an array of four nulls. And this is the first weird part. But the weird part is that even if you don't have any gamepad connector, you don't get an empty array. You get an array of all the possible gamepads that this OS is supporting with nulls inside.

[00:10:14]
In fact, if you have one gamepad, it will be the first one. So always, if you check the first item in the array, it will tell you if you do have at least one gamepad or not. And now it says it's disconnected because every time you refresh the page, the gamepad, even if it's connected to the OS, is disconnected to the web app.

[00:10:35]
So I just need to press a button or something, and now we see gamepad connected, and that means that we can run this again and see what we have. Gamepads, and now the array is not everything null, we have actually one gamepad. And if you open the gamepad, we see that the gamepad option has an array of buttons, a vibration, and axes.

[00:11:00]
If I open buttons, I have in this case, this particular gamepad has 18 different buttons, and each button is telling you if it's pressed, touched, or a current value. The problem, as I mentioned, is that we don't have an event listener like click, tell me when it's click, no.

[00:11:22]
We have to query this array on every frame and then check when that happens. And this is what I'm doing here. So let's look at that. I have a function with the name gameLoop, okay, that I'm immediately calling. This function is actually getting the gamepads and checking if one particular button, the first one, that is the X in the PlayStation format system, and then the axes, if they are greater than 50% to the right or greater than 50% to the left, and then setting some booleans in an array.

[00:12:07]
Why? Because again, you can say, well, if it's true, let's call next step, for example. If I'm going like this, I wanna go to the next step in the recipe. But if you do this, even if it's really fast, you will get five executions of the callback. So you don't wanna move five times.

[00:12:30]
So that's why I'm waiting 60 frames to check if I do have a button or not pressed. So if within the last second we had that button pressed, we go to a next step. And of course, you can play with these values until you're happy with the result, you need to do some testing.

[00:12:50]
And you say, I'm talking to the cooking app and say finish the process, go to the next step, or to the previous step. And at the end, we request another animation frame and we keep that game loop working. Make sense? So that's how that works. It's a little tricky to get used with, but here you have an example that is working.

[00:13:14]
So again, the idea is that, let me inspect this, I think the zoom is a little weird. Okay, like that. Let's open the Argentinian asado. First, it's not connected now but, all right, it is connected, yeah. See, it's connected or not? Not sure. We'll refresh, Cuz I was inspecting this.

[00:13:45]
Okay, so now you can see it's connected. And now I'm moving between steps just touching the gamepad. Yeah, it's still a physical device that I'm touching, but I think it's even more prepared on the keyboard to receive sauce, for example. But anyway, we will improve that a little bit.

[00:14:08]
But that's how you use a gamepad. Remember, it can be Bluetooth, and it can be an iPad, and that also works. By the way, this is a PS4 device, I'm not sure if you have experience with those, but it has a light. This is not the official one, I think.

[00:14:27]
No, it's not from Sony. But it has a light that it's an LED RGB, so I can change the color. This part is touch. It can also play audio. Yeah, it can play audio, it has a speaker. The question is, do we have that in the gamepad API?

[00:14:46]
No, the gamepad API is a high-level API that has only what is available on every gamepad. But can we do the rest? Can we access vibration? For example, this has an engine also, a vibration engine. Well, for that, the answer is yes and no, or the answer is light green.