Wrapping Up

[00:00:00]
>> Okay, so for this particular project, that's pretty much it. And hopefully, now you kind of understand a little bit more of the process that you have to go through. If you wanted to use data that was not made on your own, and it was a dataset that you downloaded from somewhere, usually, you would still have to resize some images and pick the shape of your tensor that you would want to use.

[00:00:23]
It's more of also a trial-and-error kind of process, but that's it. So we created a custom machine learning model. And what I want to show you next before we finish is a few extra resources. So first of all, when building your own machine learning model, I think I talked about it yesterday, but my project of air street fighter, I built it using the same kind of concept.

[00:00:49]
The model is not exactly the same with the same amount of layers, but in terms of figuring out what worked, I used the same process. I recorded a lot of live data from gesture, so it was not images, but it was live data from gestures. Instead of tensor 1D, I just experimented with tensor 2D, and I tried that, and I tried different parameters.

[00:01:12]
And when I got to a point where the accuracy was pretty good, then I kind of, like, okay, I'm keeping that model and then I'm building an interface around it. So even though we've used images here, you can use live data coming from other places. For example, yesterday when we did transfer learning and I showed you how you can use sound, and then they transform the sound data into spectrograms, you could use a similar code that we wrote today.

[00:01:42]
But first, you would also have to create the part that transforms the live data from the getUserMedia or the Web Audio API to then spectrograms, and then you could save these images. So it means that if you wanted to create a machine learning model that recognizes words that you spoke to a computer for hello or thanks, then you could represent that audio data as a spectrogram.

[00:02:02]
And then just download these images and create an audio recognition model out of images as well. So I haven't done that, but I would be quite cool. So I kinda had to pause this project, but it's actually to use brain sensor data to actually build a custom model to recognize me blinking.

[00:02:24]
So it's not using the webcam, it's not taking pictures, I don't know if you can see. I realized recently that I actually never finished that project, but I'm wearing a brain sensor on my head. It's basically getting live data from my neural activity. And using the live data, I can isolate the data coming from the sensors that are closer to my head.

[00:02:47]
And when I'm blinking, I'm basically gathering that data. And what I had is I also visualized it just to see what if I'm getting raw data from the brain sensor, do I even see a spike? It can visualize the spike when I'm blinking. And as I see that it could, then I'm like, okay, if there's a difference between my normal brain activity and when I blink.

[00:03:08]
There's a little spike, because all of a sudden, I guess your visual, I don't know how to say it. But you're not seeing any more or you're really using the muscles of your face and that changes a little bit the data that you get from the sensors, you can then use that raw data and train a machine learning model as well to recognize certain facial expressions.

[00:03:28]
So what I wanted to do then is to create a machine learning model to recognize more thoughts patterns, or at least different areas of my brain activity. I wanted to double-check that at least with blinking, it worked. And then what I wanted to do is, can I do a difference between blinking with my right eye or my left eye?

[00:03:45]
So just to show you that you can play with a lot of different types of data, here we did images cuz it's a little bit, let's say, faster to work with, but you can find different ways to experiment with data as well, and different sensors and where you get it from and things like that.

[00:04:02]
But basically, before we finish, I just have some resources that I wanted to show as well. Yesterday, there was a question about, do you know an example of TensorFlow.js used in production? So I did some research and there's actually in the TensorFlow.js blog, there's actually a few examples of companies using TensorFlow.js for production applications.

[00:04:23]
For example, LinkedIn, use TensorFlow.js. I didn't read the article totally, but I thought it was cool that your company like LinkedIn, I think it was for performance, used certain things. I can let you read it, but I just wanted to give you some examples. If you feel like, now that you know TensorFlow.js maybe you can pitch some ideas in your company.

[00:04:46]
What I really liked is that Adobe used a TensorFlow.js to enhance a Photoshop for web. So I think recently, they made a Photoshop accessible in the browser and they used TensorFlow.js to make it better. So I thought that was pretty cool as well. Photoshop is a tool that maybe a lot of people use, and I thought it was really cool that a company like Adobe uses that as well.

[00:05:08]
And I mentioned briefly, maybe in the conversation yesterday, predictive prefetching, and that is basically using navigation data from Google Analytics. If you have Google Analytics hooked up to your website, and you can get the navigation data from your users, you can use all of that data to train a machine learning model.

[00:05:28]
And then kind of have an idea of how many times certain links or certain pages are visited by your users, and you can prefetch the assets for performance. To improve performance, instead of prefetching things only when people are clicking, you can prefetch in advance and that is optimizing your performance.

[00:05:49]
So that's another example. But in general, if you want to keep in touch with what's going on in WebML, then there's a newsletter that is written by Jason Mayes, who is actually working at Google on the developer advocacy team for TensorFlow.js. So I think it's maybe monthly, yeah, monthly, he sends a newsletter with the updates and projects from the community and things like that.

[00:06:15]
So if you want to keep in touch, that's a good resource. And there's another one as well on YouTube where you have a series. It looks like the last one was a year ago. So maybe they stopped doing it, but it's still a good resource to see how far you can really get what can be done with TensorFlow.

[00:06:33]
Obviously, here, radiological image segmentation, that looks pretty cool, very different from what we've been doing here. But here, reinforcement learning, we didn't do this here, but obviously, apparently, you can do it with TensorFlow.js. And this, yes, all of these are examples from the community. Either people who've done things maybe on their job and they're talking about it, or really just open source projects that you can check out.

[00:06:57]
So there's quite a few videos in there and I can give you an idea of how you can get the knowledge that you've had, hopefully that you've gained in this workshop and then go a step further as well. So all these resources are available on the course website and there's probably more out there.

[00:07:17]
But again, now that you've learned at least the basics, the three main things that you can do with TensorFlow, then it's up to you to experiment with different ideas or look at the docs and try different types of layers or architectures and see really where you can take it.

[00:07:32]
But thank you for joining me in this workshop. Hope you've had fun and I really hope that you take what you've learned and that you really take it far. And maybe I'll see in the wild some of your projects that you build. But yeah, thank you very much.

[00:07:45]
>> [APPLAUSE]