Playing an Audio File

[00:00:00]
>> Let's start by coding some really basic examples with just the Web Audio API. And I'm gonna be using a little bit of p5.js as well just to sorta visualize just circles, squares, things like that. But primarily, I'm gonna be focusing on the audio side of things. Now, you can either go into the Web Audio demos on Glitch, and then open that up and hit the link here, maybe open it in a new tab.

[00:00:28]
I've got it open here in one of them, and it might take a little while to load. It might not even load right away depending on how glitch is doing today. So because I've just been having some issues there, I've also got it running locally. And so if you wanna run it locally, you can clone the repository.

[00:00:46]
So I've cloned it, I've copied it onto my desktop or any folder you like, you CD into it and your terminal and then you run npm install to install the dependencies. This is using npm and node.js which I explained a little bit in my other workshops with front-end masters.

[00:01:07]
But I'm not gonna go into too much detail on that here. And once you've installed it, once you've hit npm install and hit Enter and run the installation, then you can just run npm start and it should open a local listing here with all of the different HTML sketches.

[00:01:26]
And these correspond exactly to the different sketches in the glitch demos, so you can follow along either way whether you wanna go into glitch and follow along there and sort of remix there or just run it locally and open them up in your browser. So this first one is a big play button and all it is is just playing an MP3 file.

[00:01:51]
[MUSIC]

[00:01:54]
And then pausing it or stopping it. The one thing that's a little bit different about this is that we're playing a long background track so when we're playing MP3 files, we kinda have to decide if the file is a long one that we want to stream in over time.

[00:02:11]
So imagine you had a very long 20 minute composition and you wanted to play that through the browser, you wouldn't have to wait for that entire thing to download, you just wanna play it and as it's downloading, you play that audio. And so that's where we're gonna use this demo here is using the audio tag, which is the HTML5 audio tag.

[00:02:32]
And that's gonna allow us to stream the MP3 file or WAV file or OGG file or whatever sorta sound-based file you're using, it's gonna allow us to stream it through the browser so that we don't have to load it all up front. And then the next thing after that, we're gonna talk about how to buffer things for different purposes.

[00:02:49]
But this is great for like background music and long tracks and things like that. So I'm gonna go into the code editor and I'm just gonna open up this demo here and close all my other tabs. And I'm gonna just shrink this a little bit. So the basic structure of these demos is we have some global variables up here at the top.

[00:03:23]
So typically, we're gonna have a variable called audio context, and that's gonna be the variable that holds all of the details associated with the audio API itself, that's kinda the starting point. Any kind of web audio API application is using this audio context. And then we're gonna maybe some times have some variables that will be specifically around the notes that we're using.

[00:03:48]
So in this case, we just have one audio note here that we're using. We're doing everything on mousePressed. So I'm using this function from p5.js called mousePressed, which we'll get run every time we click on the window anywhere. And the reason we're using those press instead of just playing things automatically, it's because with Web Audio and with audio in general in the browser, you can't just play audio immediately upon the pageload.

[00:04:14]
That's one of those things that used to be possible. And then news websites kinda ruin it for everyone. And now we can't really just play audio right on the opening of a webpage. Browsers will force the webpage to first interacted with so you have to only initiate audio for the first time on a gesture, so maybe a keywords, press or click or some sorta forceful user gesture that initiates the audio context.

[00:04:44]
You can actually create the audio context outside of this and create all your audio nodes and only play things on most press but I like to just sometimes set everything up within the gesture so that it doesn't even try to do any audio related things until you've clicked, until you've tapped a key or something like that.

[00:05:04]
So that's why we're doing it all in the mousePressed. And that's also why there's this condition here which is to say, if we haven't yet set up our audio context, that's where this set up sorta stuff is gonna happen. Cuz once we've set it up once, if we click again, we don't wanna have to reset up the audio context, we wanna do something else like pause the audio let's say.

[00:05:29]
Just before I dive into this function, I'll just scroll down a little bit just to show what a lot of these sketches are gonna look like today. They're gonna typically have the sorta standard p5 functions which is like a setup, which is gonna allow us to set up a canvas, like this black canvas here.

[00:05:45]
And then resize functionality. So if the browser's resized, the canvas will resize, and then the draw function. This is where we can draw shapes and graphics and start to visualize the audio and start to sorta output that in a visual interesting graphic way. Right now I'm just using it to draw play button which is this polygon utility, which you don't really need to worry about how that works.

[00:06:08]
But we'll get into the draw function a little bit later as we start to visualize things. So just to focus on the mousePressed. So I'm actually gonna start from a clean slate. Why don't I do that? And worst case scenario, I might have to undo a little bit or maybe I'll just do this, keep it in another tab just in case I forget anything but I'd like to start from a clean slate rather than just talking about lines of code cuz I think it's a bit easier to sorta work through this together.

[00:06:40]
So what we want to do on mousePressed is first check if we haven't yet assigned anything to the audio context. If we haven't yet set up our global audio context, it's gonna be like this. Then in here is where we set up our audio. And the first step is to create the audio context, just like this audio context is this global variable that exists in browsers.

[00:07:10]
On some older browsers, you might have to prefix this with like WebKit audio context. But just going forward, I think in the next few years and so on, we probably won't need to worry so much about those prefixes. So I'm gonna just stick with this simple, just a variable named AudioContext.

[00:07:28]
And then what we're gonna wanna do is create an audio tag. So as I was saying earlier, the HTML5 audio tag is used for streaming audio. That's what we wanna do in this case. So we're going to do audio is equal to document.createElement, audio, this creates a new audio tag.

[00:07:49]
You also could have defined this in react or HTML or whatever and then query that with a selector and then use this in the same way, but you can just create it like this. And now what we're gonna do is set a variable on it called loop. This is optional but let's say you're doing a background track, as in this demo, you just wanted to loop all the time.

[00:08:14]
And so whenever it hits the end of the track, it's just gonna play again from the start. Another thing we'll have to do is actually set the URL for where the asset is. And this part is gonna be different depending on whether you're doing this locally, just as I am here or whether you're sorta experimenting and remixing the glitch demos.

[00:08:37]
If you're working with the glitch demos, you'll have to drag and drop your MP3 files onto their platform, and then use the URLs that they have. But here in the actual GitHub repository that I've got online, I've included an audio folder with three different test audios. And so we can just use a reference to those.

[00:08:59]
Just for now, you can pull in your own MP3 files, your own WAV files. Web Audio supports a variety of different audio formats. I tend to stick with MP3 because it has a wide support. But for some other projects, you might want to explore other formats that might be able to compress a little bit better, or be able to play in different ways like streaming a little bit better.

[00:09:23]
So to set the URL, we're gonna just do audio.source is equal to audio/ let's do piano.mp3. Again, just to comment there. And then the next thing we're gonna do is play the audio. Because this is happening on a mousePressed gesture, so it's fine to play audio. And if we try and save, because we're using this local server, it should reload automatically.

[00:09:58]
And when we click play, we're gonna hear the audio. However, we're not using the Web Audio API yet. So this audio is playing just using typical HTML5 Audio API, which is pretty old school. We can't do anything like visualizing it or adjusting the game really in a fine controlled way, we can't do any sorta synthesis with it or anything like that, can't do any effects.

[00:10:31]
And so what we wanna do is take this HTML5 audio and pipe it into our audio context that we've created up here. And so what we're gonna do to do that is to create a new source node using audioContext.createMediaElementSource. It's like the longest name, I always forget it.

[00:10:57]
So it's kinda one of these things that you sometimes have to Google or something. But this is going to allow us to create a source audio node, which is like one of these little Lego building blocks that I was talking about before with wiring things from one to the other, using our audio tag, so we pass in our audio tag as the parameter.

[00:11:18]
And now if we were to reload the page and play, we don't hear anything. And that's because the audio is now going through the audio context, the Web Audio API. And the Web Audio API has one more step in order to actually hear things. We have to actually hook it up to the speaker basically.

[00:11:36]
So to do that, we're gonna do source.connect,audioContext.destination. And this is the wiring up, so wire the source to the speaker. Let's call it speaker. Just like I was saying with the wires being a switchboard operator, and now if we play,

[00:11:57]
[MUSIC]

[00:11:58]
We should hear some audio.

[00:12:02]
[MUSIC]

[00:12:06]
One more thing we can do here is we can turn all of this off with a pause button when we actually click again the second time so that we can play in and stop the audio. So I'm just gonna do an else here. And then in here, this is where we're gonna get if we press once it's already playing.

[00:12:29]
So I'm gonna say audio.pause, I believe. And then audio context.close. And now I'm just gonna do this, I'm gonna do audio context, and just set these to know. Let's just make sure that works. Yeah, so what I'm doing is I'm first pausing the audio tag just to be sure that even if we remove the audio context, the HTML5 audio is not trying to play anymore.

[00:13:03]
And then we can actually close the audio context. And then we can set them both to know so that next time we do a mousePressed, it'll go back to this initial condition. And that's how we get back to that state we're sorta starting with.