collect-stream, from2, and to2

[00:00:00]
>> So another module I'm gonna show because sometimes this is really handy. I find that this module, collect-stream, is really useful when doing things like in unit tests. When you just have a streaming interface and you wanna test it, but you don't wanna have to write a bunch of stream glue to hook it all together.

[00:00:16]
If you just wanna save it all to an array and then do a deep equality test on your array, you can use this module collect-stream. It's sort of like concat-stream, but it gives you an array of the chunks instead of a single thing. So it's really useful for stuff like object streams or what have you.

[00:00:36]
Here's a little example you can run that will take lines of JSON from standard in, turn them to objects, and collect it all in one big array. Another package that's super useful is if you wanna implement readable streams, you could use the core readable API. But it's actually really difficult, and I don't really recommend it unless you really need to.

[00:01:01]
Instead, you can use a lot of these util packages and sort of compose what you need from there. One of these is called from to. It takes a function as an argument and the function is called whenever a downstream consumer that you're piping into needs data. So that downstream consumer will call its next function, which will trigger your function to generate some data.

[00:01:28]
So by default in Node, you get the size and you also get a function to call when you're ready to send more data. It looks like this. So if you have some messages pre-baked like this, of course. And a more practical example might come from a file descriptor, it might come from a hardware device, it might come from all kinds of places.

[00:01:48]
But here's how you can more easily assemble a readable stream rather than having to wade through the Node core docks about all of the intricate ways that you have to set it up like with the readable event. And then you have to implement a read function yourself. I like this way of doing it a lot better.

[00:02:08]
to2 is really handy when you have something, like if you're using through2, but you're not piping it anywhere, you can just swap out through2 for to2. Which is hard to say out loud, it sounds very silly. But to2 implements just the writable side of the stream. So you can pipe into it, but you can't pipe out of it.

[00:02:32]
So if you wanna implement a module that exposes an interface, for example, that only provides the writable side of a connection, you can use this package. So we had some of these already, so why don't I go in and I'll just swap out one of the through packages.

[00:02:50]
I think the line count one was a good example of that. So here we have through, but we're not piping it anywhere. So we could just as well use the to package. Select that, if I run it, then it works just the same. It's good if you do have something that's only gonna be a writable or only gonna be a readable stream and you have a module that you're either using internally in something or you're exposing it.

[00:03:24]
As soon as other people start using it, they're probably gonna try to wrap it in all of these ways that you didn't intend them to do. And then if you change it later, if you get rid of the readable side, their code will break because it was set up wrong.

[00:03:38]
There's a bunch of Gulp plugins that did this with Browserify. Cuz I was doing the same thing. I just had a through package cuz whatever. But that broke some stuff from some really poorly written plugins, I think, in the Gulp ecosystem at one point, as an example. So to2 is very handy for that.