HTML5 Media: Audio, Video & WebRTC
Though some of the concepts in this course are still relevant, overall this course does not reflect our current course standards. We now recommend you take the Complete Intro to Web Development, v2 course.
Table of Contents
Introduction to HTML5 MediaMark Boas introduces himself and describes his background using media on the web. He also gives a brief history of HTML5 media and introduces many of the core concepts.
Audio and Video Basics
SupportBefore any HTML5 audio or video can be used, you need support. Mark summarizes the current browser support matrices for both audio and video.
Containers and ImplementationThere are a few different types of audio and video containers. These containers hold different codec that are supported in browsers. Mark demonstrates how to add audio and video to a webpage and specify different media containers.
HTML5 Video with FallbackMark shows how to implement a simple HTML5 video player that includes a Flash-based fallback for browsers that don't support HTML5 media. He also discusses the different HTML5 media attributes available.
Assignment 1 - Build a PlayerEmbed video and audio into a webpage with maximum cross-browser compatibility and a simple fallback. Create the minimal possible markup for an HTML5 web page that contains both audio and video.
Assignment Media FilesThe audio and video files Mark uses in this assignment were developed years ago. Mark goes off topic for a couple minutes to show some of his first experiments with web audio and video.
Mark's ProjectsMark takes a moment to discuss a few of the media projects he works on. This includes jPlayer and Hyperaudio.
Assignment 1 - SolutionMark walks through the solution for Assignment 1: Building a Player
Tools and Configuration
Tools and ServicesThere are a lot of tools that will encode audio and video into the correct formats required by HTML5. Some tools are available for download. Others are hosted solutions.
Assignment 2 - Configure and Encode MediaIn this optional assignment, you should configure your server environment so support the HTML5 media types and encode some of your own media files.
Introduction to the Media API
Assignment 3 (with solution) - Create a Custom Video and Audio PlayerThe third assignment is to create a custom video and audio player with play and pause button. You cannot use the "controls" attribute for the play and pause buttons.
Buffering, Seeking, Preloading, and the User Experience
Preloading and BufferingPreloading requests the media be downloaded (but not played) as soon as the page is loaded. It can be configured using the "preload" attribute. The buffered attribute will return the time ranges of the media that have been completely downloaded.
Seeking, Seekable and Displaying TimeSeekable returns a TimeRange object of playable time ranges. This is often preferred over the buffered attribute. Combining seekable and buffered to display progress can be beneficial from a user experience point of view.
PlaylistsPlaylists are an import user experience element in media players. They are typically marked up using ordered or unordered lists.
Assignment 4 - Adding a Functional Progress BarBuilding off the previous assignment, you should add a functional progress bar that includes displaying media progress and allowing the user to navigate by clicking on the bar.
Assignment 4 - SolutionMark walks through the solutions for Assignment 4.
Subtitles and Playback Rate
WebVTTWebVTT is a standard that can be used for subtitles, captions, descriptions, chapters, and metadata.
WebVTT Capabilities and SupportWebVTT allows for the ability to position text and add speaker semantics. Defining chapters can make navigation easier and more intuitive. Mark also talks a little about browser support for WebVTT.
Subtitle ResourcesMark talks about a number of resources for creating and synchronizing subtitles with HTML5 media.
Playback RateA new feature in HTML5 audio and video is the ability to change the playback rate. This API is supported across most modern browsers but still has some gotchas.
Assignment 5.5 - Add playbackRate Capability to your PlayerIn this assignment, integrate the playbackRate API into your media player.
Assignment 5.5 - SolutionSolution for assignment 5.5
Canvas and WebGL
Canvas and VideoThe canvas element in HTML5 introduces a new way to draw on a webpage. The canvas is pretty powerful and can by tightly coupled with HTML5 video. Mark demonstrates Seriously.js to show the possibilities.
Creating Simple EffectsWhile Seriously.js can do some amazing effects, the process of creating an effect is simple. Mark shows how to convert a video to black and white in real time.
More Canvas ExamplesMark shares a few more examples of using the canvas element with HTML5 video and provides some additional resources. You can also use WebGL to paint video in a 3D environment.
Assignment 6 - Manipulating ColorAssignment 6 asks you to manipulate the color of your video using the canvas element. Mark provides some instruction on how to get started with this assignment.
Assignment 6 - SolutionMark discusses the solution for assignment 6 and talks a little with the audience about other uses of video effects.
WebRTC Introduction and ComponentsWebRTC enabled real-time communication of audio, video and data from browser to browser. To demonstrate this Mark invites attendees into a shared video web application. The three API's enabling this are MediaStream, PeerConnection, and DataChannel.
MediaStream APIThe MediaStream API allows you direct access to devices like the microphone and camera. Audio and video are split into synchronized MediaStreamTracks.
PeerConnection and DataChannel API'sGetting access to camera and microphone information is nice, but you need the PeerConnect API to tie those streams together with another client. The DataChannel API makes it possible to stream any data you want. It handles issues like prioritization, security and congestion control.
WebRTC Support and ResourcesMark overviews the support matrix for WebRTC and gives a number of additional resources as well as a simple code example.
Assignment 7 (and solution) - WebRTCThe 7th assignment is to capture the stream from your cameral and microphone and display them in real time. After a little discussion, Mark demonstrates the solution.
Audio and Video LimitationsWhile there have been many advancements with HTML5 audio and video, there are a number of limitations. These include issues with autoplay, preloading, short audio, and latency. Mark talks about some of these limitations and how to work around them.
Web Audio APIMark starts by demonstrating a few things he worked on for the BBC using the Web Audio API. He then starts describing some of the benefits of using this API.
Node-based ApproachThe Web Audio API is very modular. It's node-based approach allows inputs and outputs to be connected by any number of intermediary nodes.
Assignment 9 (and solution) - Using the Audio APIAssignment 9 is to use the Web Audio API to load and play audio.
Web Audio API ExamplesMark ends this topic showing a number of examples that use the Web Audio API. These include audio examples like cross-fading or visual examples like equalizers.
Creative ExamplesMark showcases a few creative examples that use many of the HTML5 media features including ways the BBC has recreated sounds from the 1960's by simulating radiophonic equipment.
Audience QuestionMark answers an audience question about how he synchronizes and stores the transcripts that accompany the video/audio files.
Wrap-upMark ends the course with a couple more questions from the audience regarding his Hyperaudio application.