Frames & Parsing Message Buffers

[00:00:00]
>> Open this back up. So now we're running on 8080. Again, come over here write our names as Brian. I hit submit obviously it's not gonna I'm not doing anything with it yet. But if you come over here, look at what you're getting back here from the server.

[00:00:18]
Something, right? It's an encrypted buffer, right? So we actually have to go and we have to decode everything that we're getting back from the client, because it's being sent in this particular binary format that's been encoded. Yeah, so these are called frames, right? And if it's a like a really large output, it'll actually it will definitely break these over multiple frames, right?

[00:00:47]
So if you send it back, 100 pages of JSON I'm going to break that over multiple frames as well so that it can send those packets faster. It makes the implementation harder to deal with but it makes it a very flexible way of communicating because we can send videos over WebSocket for example, right that can be.

[00:01:04]
Can you broken down, written two frames and then sent over across a WebSocket connection, it makes it really good for stuff like that. So let's go dissect a frame for just a second. Let's scroll down here and you'll see this. This diagram that I took from think this might be this spec.

[00:01:28]
I don't remember. Yeah, this is actually from the protocol spec. So if you got way too much time in your hands and you're super bored and you want to be more bored, go ahead and read this. I did read part of it. Some of the stuff in here.

[00:01:43]
Yeah for example, this is the string I was telling you about. It's just right there. Someone's like, okay cool. This seems like a good idea. We're just putting this magic string in here. Someone's gonna like tweet at me and say like, this is why this was a good idea, Brian, I'll be like, okay, that's fine.

[00:01:57]
I believe you cheap jokes. Okay, so this is like what a frame ends up looking like. So these are individual bytes in here, bytes bits. But the important parts here is one we have an opcode here. So there's different kinds of frames that you can send there's like an opening frame there's a data frame or I think it's called a text frame.

[00:02:27]
There's a closing frame when you're done and you're closing it to like significant the fate of the server or the client, hey, I've decided that I'm done talking to you. Here's a closing frame. All that stuff is stored here in the opcode. Here we're giving it a payload length of how long of how much data to expect from it.

[00:02:47]
If you need more than you can actually extend here into this extended payload length. You have to give it a masking key. So note, WebSockets themselves do not require masking which is basically you use like this masking key to encode. What you're sending over, however, the browser always masks it.

[00:03:06]
So we actually, we're gonna assume this is always set to one in our particular implementation. That's the masking key here. And then here, everything after this is just payload data, right? Stuff that we're sending to the client. Not super important that you understand this. I'm just Just kinda of letting the concepts that I'm about to show you in code, right?

[00:03:25]
This is, the raw information that's being pushed over your socket. Okay, so the only opcodes of your care about one which is n and we care about eight is the closing frame. We're just gonna ignore a lot of this, to be honest with you. We're just gonna implement that what we need to make our app work and we're gonna ignore the rest of it.

[00:03:51]
But something like socket, io will handle all of this stuff for you, right? They've actually gone through and thought through all of these permutations, okay? So let's hop back over to our code and we're gonna go look at parse message. This is the most complicated part of this.

[00:04:14]
So we're gonna give it a buffer. So basically what we wanna do at this point if we look here. We wanna take this, and we wanna translate this back into text. That's the goal of taking a buffer, and making it back into text. Just like we did with the object to response where we made it a buffer, and sent that over the network.

[00:04:33]
We're now going the opposite direction. Okay, so the first thing is we're gonna read the first byte, and this is gonna be the opcode, right? So you're gonna notice here we're gonna see a lot of individual hands and we're gonna see a lot of these things. So the and this is a logical end bitwise operator.

[00:04:56]
And what a bitwise operator do if I have like a full byte 0b10010011, right? And I say and 0b00010010, this does a logical and between the two of these things. And what that actually does is this is really easy to see basically look like where does it have both one in one of these, right?

[00:05:29]
So here, the product of logical and here would be 00010010. Because you can see it's where these two matchup. You'll sometimes see or you'll sometimes see other ones like I'm trying to remember. I don't do a lot of it was operating like I always have to look these up whenever I'm teaching these, but you'll see these down here in here.

[00:05:58]
These are shifting operations we actually just moving things over. And then he had we're doing a bitwise between the first byte that we're getting back from the frame, and then hexadecimal 16 which I think you can represent as it's just representing that as this two, four eight. Anyway, whatever I'm not gonna translate into binary but suffice to say this ends up being 16, right?

[00:06:28]
Hexadecimal, O x f This is ends up being the number 16. They're just different base numbers right? Binary is base two hexadecimals base 16. And you and I are most used to writing hexadecimal or so just decimal base ten. Not a class in binary but I just wanted to let you know the stuff that's that's going into here.

[00:06:50]
So opcode, we're looking here for opcode eight, right? This means that connection this close so we return null and we're just saying cool. We're all done here, wrap it up. The connection is now over. If we see any other opcode besides one, we're just gonna return because we don't know what to do with that, aka we're not gonna implement it.

[00:07:12]
So we're only care about text frames here which is opCode 1. Okay, we read the second byte here. And this is basically saying we're looking for is is this a mast number, right? A non mass number you would just be able to read the numbers directly out of their mask.

[00:07:30]
You'll see down here that we have to use a mass to convert from one thing to another. So we're looking for is it masked, if it's not masked, then we're gonna say, well, you're obviously not a browser, so I'm just not gonna deal with you. So this is where we're saying here we only care about masked frames from the browser.

[00:07:48]
Here where to get the maskingKey. And then down here, we're checking the length of how long it is and we're saying We don't care about anything that's big, right? So we're just doing small frames here, but this is where you would handle long frames if that's what happened.

[00:08:07]
Okay, and then at this point we're just gonna start reading stuff out of the buffer and piece by piece convert adding it to an unmasked number. This is the mask here. The mask down here, which we're using a masking key. Suffice to say, we have like a four number mask and we're just looping over the position because you have to kind of loop over it continually.

[00:08:32]
And we use that to convert that from a buffered mask stream into just a normal buffer here. And then at the end here after our for loop is done. This is another bitwise operator there, this is another shifting operator here. Same thing shifting operator here and a logical and there.

[00:08:59]
Between doing all that you end up with at the end like a byte that can be written out as UTF8. Yeah, that's what we're doing here with the response. Yeah, or you went to a truther there. And we're writing that to our sponsor, and eventually we're gonna end up with a string of something that we can actually understand.

[00:09:17]
So we take that buffered binary string and we say to string UTF8, which will end up being a JSON string, which we then do JSON.parse, right? And at the end of this entire parse message thing, we'll end up with an object of what we got from the WebSocket.

[00:09:37]
So, how much of that do you need to understand like, just zero, right? It doesn't nothing there's actually really critical to understanding here. One, I just think it's super interesting and two, I think it's cool for you to understand what's actually going on over the wire there. And to me, it's interesting If you're doing like socket.io, or anything like that, somewhere in JavaScript code, this is happening, right?

[00:10:03]
So this is something that I had assumed forever that was just being handled by the c++ layer or some part of the the node engine natively. It's not, some jerk just like me wrote this in an open source library and released it, all right? I think it's cool, okay?

[00:10:23]
Does anyone have any questions about this? I will say that I don't really understand it a whole lot more than I'm what I just told you. I dug a lot into this but it's just, there's an infinite depth here.