How Audio Works

[00:00:00]
>> So first I wanted to run through some of the basics of how digital audio works and how audio works in general. This is kind of a prerequisite to understand some of the libraries and some of the things we're gonna be using with code, is to understand how do we really perceive this kind of audio and what does it really mean for sounds to be happening around us.

[00:00:22]
So when we talk about sound we're talking about just waves through the air. And by waves I mean changes in the air pressure and changes in the air molecules. So for example, let's say you're listening to a speaker and the speaker is pumping. It's going in and out in pumping, and it's that pumping motion that creates this propagation of air molecules through the air that eventually hit our ears.

[00:00:46]
And it's sort of undulating and changing in different ways, in different frequencies. Hitting our ears at different times and things like that. Another way to visualize this is this little animation here. On the left, we have a speaker which is pumping in and out some sort of music.

[00:01:03]
You can imagine maybe you're at a dance club or something and it's a lot of these kick drums. And on the right we have an arm and mechanical arm that's attached to that motion. And every time it goes in, it draws up every time it goes out, it draws down, etc.

[00:01:18]
And what you end up with is a graph that looks kind of like a sign wave, just going up and down in this smooth undulating motion. And when the speaker's turned off, there is no more motion there and so it just draws a straight line in the middle which means that it's silent.

[00:01:34]
That's kind of how these sound waves work. And so when we say the word waveform we're just talking about this kind of graph almost this form and the shape of the wave that we're trying to visualize. But usually when we think of waveforms, we think of it more like this.

[00:01:51]
A music track might be a couple minutes long. And so if we take the waveform of this music track and we zoom it all the way so that we can see the entire waveform, it might look like this where you can clearly see that there's some peaks and some quiet parts and some loud parts and some different sort of frequencies going on there.

[00:02:11]
But at the end of the day even this music track this long large waveform if you zoom in close enough it's still just like the earlier shapes that I was showing you just a line that's going up and down on some graph. The way it works is that the horizontal axis is time.

[00:02:28]
So maybe measured in seconds or milliseconds or however long your music is. And the vertical axis is amplitude. The amplitude refers to the size of the wave the height of it, how loud it is. So the higher this is, the louder it'll be. The closer it is to that straight line that I was showing earlier, than the quiet it'll be.

[00:02:49]
These values in amplitude always go from positive one to negative one. They don't go really higher or lower than that everything is normalized into that range. And the time here I've just put 0 to 1 milliseconds but that's pretty arbitrary it could be in between you know 50 milliseconds to 51 milliseconds or it could be measured in nanoseconds or some other measurement of time.

[00:03:11]
And again when the line is straight across on the 0 amplitude it means we have total silence we can't hear anything. One other concept to understand here is frequency we're going to be talking about this when we start to visualize and try and manipulate the audio. But sound waves can be either high frequency which means they change at a very fast rate, or they can be low frequency sounds, which means they changing very slowly.

[00:03:36]
So that it's as if the speaker is pumping very slowly or very quickly. And those two differences will give us a sort of different perception of that audio. So when sound is fluctuating and changing very rapidly we have a high frequency sound that's perceived by us as something that's high pitch like treble like cymbal crashes or chimes or very high pitch sounds.

[00:03:59]
Whereas something that's more like a sub bass, like a kick drum or even like a guitar pluck, those are low frequency sounds that are changing at a much slower rate. And that's where we start to be able to analyze and start to be able to manipulate this audio is just by tweaking or looking at the frequency of the sound that's coming in.