Object Streams

[00:00:00]
>> We just left off on an example using duplex streams. The next thing that I want to talk about is object streams. So thus far, we've been dealing with buffers and strings, but it's also possible to use arbitrary objects. So, I think this is most useful when you have something like a parser.

[00:00:21]
And you want the parser to produce objects from strings. And you also want it to sort of respect back pressure. So you can have a whole pipeline of operations that happen in a streaming fashion, without having to serialize, and then deserialize a bunch of times. So, it's kind of, it can be useful for different sorts of optimizations like that.

[00:00:46]
To use objects streams, you have to enable them with a special Object Mode colon true in camel case. Or if you use through2, you can use this method through.obj, and that does it for you. So, these are equivalent. Of course when you pipe an object stream to somewhere else, that destination stream also has to be capable of object streaming mode.

[00:01:15]
So, you'll have to enable it separately. Yeah, why don't we go ahead and just poke around a little bit with object streams. So, I'll make a new file obj.js. We'll use through2, and we'll just read data from standardin for right now. I'm gonna cover some other modules that will make some of this stuff a little bit more flexible and powerful later, but for now, this, I think will do the trick.

[00:01:43]
So there are two ways of doing it, I'll just type them both out. So you can do objectMode true. This is the only way if you're using the core stream.transform. Or you can do, Or you can do just through.obj, is a nice little shorthand for that. And then in our write function, We can Turn those buffers into objects, why not?

[00:02:15]
So why don't I keep a running tally of bytes, and I'll just output the total number of bytes that we've seen so far in the message. So can do, just like normal, just like how you normally use the next function or this stop, push to produce values into the output side of the stream.

[00:02:38]
This time I'm gonna include length, buffed out length, and then total will be size plus equals about that length. You can put whatever you want in here. So this is gonna produce values, but we haven't read the values anywhere yet. So to read the values, what we can do is pipe into another object stream.

[00:03:04]
So we'll call this, better name these write functions a little bit differently. So this one will be write1, write2, sure. Okay, so the next one we make is called write2. Now we're gonna get objects as input not buffers. And we don't have to produce anything in this one, because it's a terminal stream.

[00:03:26]
There's another module called two two that I'll get to in a moment, for doing that without a transform stream. But for now, I'll just say console.log obj equals obj. So now if we run this program, we should get a running tally of the total amount of data that we've seen so far.

[00:03:45]
And the other property in the object should be the size of each piece of data that we write. So if I run it, lets say one. We get length of 4, because it includes the new line at the end, O-N-E new line, and total 4. Two is also length 4 but now we've seen 8 bytes.

[00:04:05]
Three is gonna be 6 bytes, and we've seen 14 bytes so far and so on. So, just kind of a fun thing to do. One other thing that we haven't used yet is through, and also the core stream, core transform constructor, can take end function. In core, it's called flush.

[00:04:28]
It's the same thing. So, why don't we have a little function that can happen at the end? So now maybe instead of just calculating the stuff from objects, we can do something at the end here as well. So maybe our object stream won't tally these itself, but we can keep that in a piece of state.

[00:04:57]
So I'm just moving some stuff around to show. And then at the end we'll say, size equal size. Make sense? Okay, so when I run this program now, it should just printout the number of bytes, I mean, they're better ways to write this that don't use object streams.

[00:05:20]
But this is just an example of how to use object streams. And later, we'll use some modules that deal with object streams for us, that are a bit easier to use. So whoops. Right, shouldn't be called buff, it should be obj. Now I do Ctrl D to end it and I get the total link.

[00:05:47]
So that's just kind of an example of the end function. You can also listen for the end event, because transform streams are both readable and writable. So there's an indent present. The end function is a little bit easier sometimes. Another important thing to note about readable streams is that, because you're pulling from them, if you don't hook them up to an output, they will just sit there and not do anything.

[00:06:14]
So you'll never get an end event if you don't actually read the data. Because they're only pulling from the data source when you tell them to pull, something to keep in mind.