Checking the Callback Queue

[00:00:00]
>> Will Sentance: And now, we must introduce, cuz what runs next? The whole time in node, before any line of code, any function gets run, node is going with the help of libuv. Let me check, is there any code, I mean this is a much more pure detail, I'm just trying to want to introduce it for now.

[00:00:23]
Is there any code that we've commanded to run? Any code that's currently sitting or running on the cool stack? Or is there any global code to run? Those two conditions, if the answer is no, then start checking the queues. Ha that's like a Frank Sinatra reference. Start checking the queues for the-

[00:00:48]
>> Speaker 2: [LAUGH]
>> Will Sentance: [LAUGH] I like that, too. Raise your hand if you got that reference. A small subset. Do you want me to sing it?
>> Speaker 2: Yes.
>> Will Sentance: Start, [COUGH] Okay. [LAUGH] Big mistake. All right, and this checking of the queues. Once all global code or any code we're running is finished this checking of the queues, it has a built in feature that does that.

[00:01:14]
Built by libuv, and it's in the browser version of JavaScript as well. But there it's going to the browser. But we don't have the browser here, so we had to build it from scratch again using libuv. And it's known as what?
>> Speaker 2: Event loop.
>> Will Sentance: The event loop, the event loop.

[00:01:33]
It is the node with the help of the abuse a feature that looks at what's in the queues sees what's being awaiting to run. I'm ready to run I'm ready to be auto run by node, check there's no code currently running. And then grabs from the callback queue, we're going to see there's a quite a few queues you get to choose from.

[00:01:56]
We'll see a few of them after the next break. We may even hint at most of them. There's a total of six cues that it gets to choose from. Nice, we're probably only going to see two or three of them. Because most of them it don't just use most of the time.

[00:02:11]
But for now, one queue sees what's in there, it checks the only code that left to run? No, we've got rid of do on, we finished running the last time. And at whatever would be 2.5 milliseconds cuz we are ready to run it in 2 milliseconds. But we're still inside this function here.

[00:02:31]
And so do a new batch the next running in for the next batch as it's 4 kilobytes had to sit waiting, but at 2.5 milliseconds it goes. Yes, you finish running to a new batch, the previous running of it in which you were running clean tweets all finished.

[00:02:47]
And so grab the next running of to a new batch. And put it on the call stack. And run do own new batch again. Create a brand new every always as a memento. Awaken ourselves. Michael was on his phone taking on a brand new execution contest. Did we make it.

[00:03:13]
>> Speaker 3: No.
>> Will Sentance: Who made it?
>> Speaker 3: Node.
>> Will Sentance: Node. Good, there you go. Execute it, execute it and we're not going to go into the details. Its input is the next batch right, it's the second batch of from the stream. The next batch is 64 kilobytes the second one this was the first batch The 64 kilobytes the first one, there's the second one.

[00:03:39]
Clean it, append it to clean tweets and at the very end, we would then want a JSON pass that clean tweets. Look at this, people. We're lining this up beautifully. We're lining this up such that. We were able to start passing and working on cleaning tweets as we go.

[00:04:00]
No longer are we gonna have to wait for all the data to be finished at once. Now, we can pass in the data, start cleaning it, move on. Pass in the data, start cleaning it, move on. This is Node's favorite feature. This is the feature that allows it to start working on things before it's got all the data in hand.

[00:04:27]
But to work on it in terms of it being batches or chunks. Of data, and by the way all background features or most of them technically, not all of them. Many of them where your flowing data in and out are really just batches chunked, chunkified set of data.

[00:04:49]
And if you see the HDP one. Well for a tiny little thing like a get request, it's just one chunk. But if it's sending back a bunch of images, or a movie, or whatever like that, then it's gonna be sent in chunks out of the socket, using this model as well.

[00:05:07]
Okay, and how do we own a chunk being complete, get access to it, using nodes A vent system. That whenever a chunk gets in it auto triggers a data event. We get to work on that data and continue in the background bringing in the next chunk. And when it's done, work on the next batch.

[00:05:34]
And by the way you people, I will say this, I contrived this so that it took a little bit longer to clean the data than it took to bring in the next batch. Just so we could see a function being blocked from running. So I could introduce to you the fact that we need to queue up functions to run in JavaScript.

[00:05:59]
I wanted to start introducing that to you now. Because it will make the final slides, I think the final slide's coming up. Yeah, the final slide more accessible, if we've already been introduced to the notion that we have to queue stuff up. Auto run functions have to queue up before they're allowed to run in node, a little bit contrived.

[00:06:15]
I actually reckon that there's no cleaning function that would take longer than the time it would take to get 64 kilobytes of data in. That's gonna take longer than any cleaning function, I would suggest. Which would mean that actually we would end up, this is a little bit of a sidebar, people, it's not too important.

[00:06:36]
But if you said this is the time, what was it that we said 15 seconds to get in all the data and then we were pre. This is the time to get all the data in 15 seconds and then we were, once the data came in cleaning. For we said 20 seconds, but actually reality it would be more like let's say 10 seconds we were cleaning it a total of 25 seconds.

[00:07:03]
If instead we're doing this where we're importing, cleaning, simultaneously importing, cleaning, importing cleaning, then we would finish. At this point we'd finish importing all clean that last little batch and we would be finished in just 15 seconds.