Zlib Core Streams

[00:00:00]
>> Other fun things to do is to poke around at the zlib module, which is also part of Node core, so you get this when you do require zlib. And there's all kinds of methods for doing inflate and deflate, which are used in HTTP servers or, I guess, yeah.

[00:00:19]
So I guess createUnzip can sort of automatically detect whether it's inflate or gzip. But what we can do right here is to use createGunzip to unzip something on the fly. And maybe we could take the hash of it after that, I don't know, something fun like that. And we can verify that our program works like we would expect by using the command line alternatives to doing that in Node core.

[00:00:47]
So from day1, we had this file, I guess I deleted it, but we had a file that was gzipped. I'll just make one really quick. So do gzip from standard in, from file hello.txt, and then hello.txt.gz, there we go. Just to verify that it has the correct output, there we go.

[00:01:14]
If I cat it, it's gonna look real nasty, there we go, it's some binary garbage. So we can write a note program now called gunzip.js that uses zlib.createGunzip to decompress that input. So I like to take standard in, you can of course use fs.createReadStream if you wanna read from a particular file.

[00:01:43]
But this is a nice flexible way to do it as well. And we can pipe that to standard out. So this is pretty much what the gunzip utility on the command line can do, although that one's faster, but this one is JavaScript, so that's nice, too. So if we take our file hello.txt.gz, we get the plaintext version, which is very handy.

[00:02:07]
Let's go ahead and add a hash as well, why not, to our little pipeline, I think that'd be fun. So we'll load createHash from, I guess for consistency, I don't have to do zlib.createGunzip, I could just createGunzip. And then we'll require crypto.createHash. And now we'll pipe into createHash('sha512').

[00:02:38]
And now if I run this program, it should first take the compressed gzip input, it should decompress it, and it should pipe it into streaming SHA 512. And it should print out the, I'll set it to hex encoding so that we can look at it and verify on the command line that the program is working like we think it ought to, okay, does this all make sense?

[00:03:02]
So again, standard in comes in, it's a readable stream. It goes into createGunzip, and the compressed input turns into uncompressed output. Then we compute the SHA 512 hash of that input, and then we print the resulting hash. And all of these happens in a streaming way, except for at the very end because the hashing algorithms have to buffer up the input.

[00:03:29]
But it's not like you're consuming a lot of extra memory, because the hash is done in place with a fixed chunk of memory. So if we run this program on our compressed input now, we should get the hash back that should match the original. So let me just run that a different way, okay, so here's our hash.

[00:03:51]
And that should match up to the shasum for 512 of hello.txt, and it does. And if we wanna do the same thing on the command line that we just did in our Node program, it would look something like this. So we could gunzip hello.txt.gz and pipe that to shasum -a 512.

[00:04:09]
And we get back the same hash, so any questions? These are core streams that are good to play around with, good to practice, good to know about. Comes up now and then when you're building stuff. I think it's really fun way to build programs, instead of just building these really big and clunky, enterprisey things.

[00:04:33]
Just throw down some shell scripts, throw down some little pipes and stuff. So I'm gonna add, what was that example called, gunzip.js as well. And then maybe we can take a little break so people can play around with these examples. And if you have any questions, be a good time.