Table of Contents
IntroductionMaximiliano Firtman reviews the topics discussed throughout the course. The code and demonstrations will cover permissions & security, sensors, speech, OS integration, and other hardware-related capabilities. The demo application and source code can be found on the course website.
Web Capabilities MaturityMax defines a capability and shares how they are categorized by maturity. Green APIs are available in every browser on every platform. Light-Green APIs are only supported in some browsers but are mature enough to be used in production.
Origin TrialsMax introduces origin trials, which allows developers to provide experimental APIs to a wider range of users. Rather than requiring users to enable a flag in their browsers, developers embed a token in a header or meta tag, automatically enabling the feature in Chromium-based browsers.
Unsupported & Future APIsMax concludes the summary of capability maturity levels by discussing the dangers of using red-labeled capabilities. These capabilities have little browser support and may not be added for months or years.
Web Capability ResourcesMax shares several resources for understanding how to use capabilities and researching their status or browser support.
Basic CapabilitiesMax shares several basic web capabilities that this course won't cover. These include PWA APIs, web authentication, web storage, and background APIs. These APIs can be found in other Frontend Masters courses.
Permissions & SecurityMax describes how browsers will require and prompt the user for permission to use specific capabilities. These security measures are in place to protect users from malicious use of capabilities that can potentially exploit or expose a user or their data.
Permissions Policy & APIMax introduces the Permissions-Policy spec, which allows a header to be set to turn on or off a capability for a specific origin or across multiple origins. This is useful for turning off capabilities and eliminating a malicious actor's ability to execute code in a browser console.
Sensors, Geolocation, & Input Devices
SensorsMax introduces sensors and explains how the APIs will vary based on a device's capabilities. Available sensors may include an accelerometer, gyroscope, magnetometer, proximity, and light sensors. Global DOM APIs still exist for accessing sensor data. The Sensor API is the new promised-based API which is still gaining browser support.
Sensors APIMax demonstrates how to use the new Sensor API. Each API has a constructor for creating a sensor object and events for reading and handling errors. Some APIs are behind a flag because the API is still experimental.
Geolocation APIMax introduces the Geolocation API, which uses several metrics to determine a user's location. The metrics may include data from GPS satellites, WIFI networks, and other location-enabled devices.
Geolocation API PracticeMax uses the GPS API to display the latitude and longitude information. A condition first checks to see if the API is available. Functions for receiving the position value and handling errors are created, and configuration options are reviewed.
Location AccuracyMax discusses some caveats with location accuracy on iOS and iPadOS. Users specify whether they want web applications to use precise or imprecise location data. This will apply to all web applications, and there is no API for determining which mode the user selected.
Gamepad APIMax introduces the Gamepad API. It works with both USB and Bluetooth controllers. Since the buttons and controls vary greatly across controllers, a game loop must be created using requestAnimationFrame to watch for input and determine which button was pressed.
Cooking Masters Project OverviewMax walks through the Cooking Masters application. The application allows users to follow recipes without using traditional computer input controls. Users can use gamepads, audio commands, and other capabilities for navigating the interface.
WebHID APIMax demonstrates how to use the Web Human Interface Device API, which is built on top of the widely adopted HID protocol. Permission is granted through a browser dialog and the API provides low-level control with the selected device.
Speech, Voice, & Camera
Speech Recognition APIMax introduces the Speech Recognition API. Once microphone permissions are granted, an application can use the API to detect words or phrases in a specified language. A confidence value is also returned as the speech is processed.
Speech Synthesis APIMax demonstrates the Speech Synthesis API, which speaks a string of text through the computer speakers using the available voices in the operating system.
Shape & Face DetectionMax demonstrates the Shape Detection API, which provides methods for detecting barcodes, text, or faces using a device's camera. The Advanced Control Camera API gives the application control over pan, tilt, and zoom and can be used to adjust the input stream when live streaming or taking photos.
Augmented RealityMax explains that augmented reality can be experienced in web applications using the WebXR API. Since this API is a low-level API, developers often prefer to use a higher-level solution like Model Viewer
Screen Wake Lock APIMax discusses the Screen Wake Lock API, which allows applications to prevent the operating system from locking the screen.
External Hardware & Devices
Web Bluetooth APIMax introduces the Web Bluetooth API, which provides low-level communication support with connected Bluetooth devices. The API can be difficult to use because instructions must be sent to devices in binary. Developers can find documentation with individual vendors or through third-party libraries built on top of the low-level APIs.
Bluetooth Light DemoMax demonstrates how to use the Web Bluetooth API. Bluetooth lights are connected to the Cooking Masters application. When a time finishes running, the lights flash and the gamepad rumbles.
Web Audio & Web MIDIMax explains that the Web Audio API can generate dynamic or 3D audio. It's another low-level API that third-party libraries like SonicSocket simplify. The Web MIDI API is also discussed in this lesson.
Web Serial & Web USBMax discusses the Web Serial API, which communicates with serial devices connected to a user's computer via USB, Bluetooth, or another serial connection. Another low-level API, Web USB, is also discussed in this lesson.
Vibration, Battery Status, & Idle DetectionMax reviews the APIs for vibrating a device, reacting to changes in status with the battery, and determining the idle or lock state of the user and screen.
Web NFCMax demonstrates the Web NFC API by reading data from an NFC card and an Apple Airtag. The data transmitted depends on the tag/device. Developers process the data manually using the "reading" event.
Progressive Web AppsMax demonstrates the features and how to install progressive web applications. PWAs behave like native applications by running outside the browser in a native window. The history of PWAs is also discussed in this lesson
File & URL Protocol HandlersMax walks through the metadata, which can be added to a PWAs manifest file allowing the application to handle specific file extensions. The differences between PWAs and technologies like Electron or PhoneGap are also discussed in this lesson.
Web ShareMax looks at the Web Share API, which triggers the native sharing dialog in an operating system. An application can also be configured as a share target when a specific file format is shared. This adds the application to the operating systems share menu as a possible destination.
Contact Picker, Fullscreen, & Payment RequestsMax covers the Contact Picker API, Fullscreen API, and Payment Requests API. The Fullscreen API is used by the Cooking Masters application to launch into fullscreen mode when viewing a recipe.
Banners, App Badges, & ShortcutsMax shares use cases for a few additional APIs. The Installed Related Apps API is useful for detecting if a websites related application is installed so users can continue using the site in the application. The App Banner API is also useful for prompting users to use the installed application.