Iterators and Observers

[00:00:00]
>> [MUSIC]

[00:00:04]
>> Jafar Husain: What's the difference between an array and an event? This was a question asked to me actually five years ago when I was at Microsoft when this technology was just being developed, and the answer absolutely blew my mind. It totally changed the way in which I approached asynchronous programming, and I hope that you guys feel the same way after this presentation in this class.

[00:00:21]
So events and arrays are both collections. I want you to start thinking about them in exactly the same way. Even though they have very different interfaces today in JavaScript, it doesn't have to be that way, and I don't think it will be that way for much longer. So why don't we just program these things the same way if they're both collection?

[00:00:36]
Why do we program slurping data out of arrays differently than we would events? We should approach them the same way. How do we get to this place? How do we get to thinking about these things so, so differently when I just showed you that we can think about them in a very similar way?

[00:00:51]
Well, to answer that question, you got to go all the way 20 years back, went 20 years back to 1994. It all comes from a little mistake that was made almost 20 years ago. Who's got this book or read this book? One guy back there. This was a popular book.

[00:01:06]
It was a book, the idea was of 20 years, ago you had a lot of software engineers, and they got together, and they said, well, people are solving the same problems again and again, all through the industry, but they're solving it slightly differently. What we need to take these common patterns in software and put them in a book, and that's how we got the Design Patterns book from the gang of four authors there, those four authors.

[00:01:26]
And that was a tremendously influential book. It was targeted at C++ developers and Smalltalk developers, but nowadays, this notion of design patterns is still popular in the Java community. It's probably less well-known in the JavaScript community, but it was very influential, and it defined a lot of the patterns that we use today in computer science.

[00:01:42]
Now, there was a picture at the back of this book. Don't worry, we're not teaching all this stuff today. I'm actually just gonna focus on two of these things. They are actually showing how all these patterns were interrelated. But today, we're only gonna focus on two of them, the iterator and observer pattern.

[00:01:57]
So the iterator pattern is actually something that's coming in JavaScript 6, which we'll talk a little bit of how that's gonna work. Now, they did not think these two patterns were related. They didn't even draw a line between them, if I go back there. They don't even think that this pattern are related.

[00:02:11]
You see the observer way out there, but I'm gonna suggest to you, they are related. They're actually deeply related, and understanding their relationship is key to becoming an expert in asynchronous programming. So here's how an iterator works, it's very simple. You have a producer and a consumer. And the consumer requests information, one at a time, from the producer until one of two things happens.

[00:02:36]
The producer says, I have no more information for you, or the producer says, an error occurred. Does that make sense? So how this looks in JavaScript is in JavaScript 6, the next version of JavaScript, you're gonna be able to walk up to any array and request what's called an iterator object.

[00:02:52]
So think of this is a way of a function returning multiple values. So, an iterator object has a very simple interface. You can call next, and you'll get out this little object, which gives you the value that you requested and also gives you a little boolean indicating whether or not there are more values for you to get.

[00:03:08]
So here, I got one, but I see that we're not done, so I'm gonna call next again, done now, I have 2. And I see we're still not done, so I'm gonna call next again one final time, 3, not done, call next. Finally, I get a little message that says, you know what, we're done.

[00:03:25]
So go back to that consumer-producer relationship. I'm the consumer, in this case, and I'm just pulling values out of this producer, which is the iterator. Pulling values out until the producer says, I'm done, no more information for you. So the gang of four to find this pattern in the 80s.

[00:03:42]
I think 80s early 90s, and whereas now, it's actually just making its way into JavaScript. So it's a nice convention for having any data structure. I might have a set or a map or a vastly different data structure. It doesn't matter what it is, I can walk up to a data structure, and using this simple protocol, I can pull all the elements out of it sequentially one by one.

[00:04:01]
Does that make sense? That's why they defined this particular design pattern, so that you could walk up to any particular data structure. You don't know anything about its internal representation, but you can still consume the elements inside. And this is actually the underlying technology that enables the new for-of loop.

[00:04:15]
There's a new loop coming in JavaScript 6, specifically for consuming this, so you don't have to write all of this boilerplate code. You can write what just looks like a simple for loop. So Map and Filter and ConcatAll, in addition to being implemented over an array, can also be implemented over an Iterator.

[00:04:29]
Let's just think about that for a moment, right? I can write a map function, which given an iterator returns another Iterator. And every single time it pulls a value out, it applies that map function to the value and then returns the translated value. So you can apply a Map and a Filter and a ConcatAll.

[00:04:45]
ConcatAll would be a two-dimensional iterator, you'd flatten it into one dimensional iterator. All the operations I've shown you thus far are actually at their root. Probably, they can be expressed over iterators. Now, I'm gonna show you another pattern that the design pattern guys expressed, and this was all about you wise.

[00:05:01]
They were concerned about how is it in the user interface, if you have a change to a model. How do you communicate to the view that something is changed or vice versa? How does a view communicate to the model that a user wants to do something? And they define this pattern called the observer pattern.

[00:05:15]
Now, probably most people in this room are familiar with the observer pattern, because it's something we in JavaScript use everyday. Because, it just so they still find a way that you could add a function to some data producer. And instead of the consumer pulling the data out one of the time, instead the producer pushes data action, right?

[00:05:35]
That's how an event works. You hand me a callback, and then when I wanna send you information, I'm the producer, I push you information one item at a time. Does that make sense? So when one of these patterns in the iterator pattern, the consumer is in control, the consumer is pulling values out of, it decides when it gets the next value.

[00:05:53]
But in the observer pattern, the producer is in control. The producer decides when you get the next value, cuz it calls your callback and pushes it at you. Does that makes sense? So it's really, these two patterns are about doing the same thing, fundamentally. They're about a producer getting a consumer items one at a time.

[00:06:09]
The only difference is who's in control. But the design patterns guys made a little mistake. So this shouldn't blow anybody's mind. This is just what happens, right? I move my mouse around, and we get the results. They did a little mistake.
>> Jafar Husain: They failed to see this connection, that at some deep level, these things were both about the same thing.

[00:06:29]
We're both progressive about progressively sending information to a consumer, and they didn't see that the iterator and observer pattern were symmetrical. And so this is how you understand the observer pattern, if you already understand the iterator pattern, right? The observer pattern iterates you, that's what's happening. At the observer, the producer is iterating you, right?

[00:06:46]
So they missed this symmetry, and as a result 20 years later, we think about asynchronous programming very differently than we probably should. There is a subtle difference between the iterator and the observer pattern as they outlined it in design patterns. Remember what I said about the iterator pattern.

[00:07:07]
You can pull values out until one of two things happens. What are those two things? Can anybody tell me?
>> Speaker 2: Error and then array.
>> Jafar Husain: Error, and it's done, right? No more data, or the producer throws. That means if I call next, they could throw, right? And that's how I would know that an error occurred.

[00:07:23]
But with the observer pattern, they failed to add any of those in any well-defined way of the producer expressing to the consumer that, A, there was no more data, And B, that an error occurred. Let's go back to our dawn of that listener example, right? You pass one callback to this thing.

[00:07:41]
The only thing that you can receive, the only message you can receive from the producer is here's some more data. And why did they do that? Well, they were very focused on the event used case, they were very focused on solving for UI events. And the thing about the UI events is there might always be another one, right, because somebody could always click another mouse click, they could always move the mouse again.

[00:07:59]
And so they weren't really thinking that, you need this notion of completion, because they work very narrowly thinking about how the model events. But today, 20 years later in JavaScript, we deal with push streams that end all the time. Anybody ever worked with a node stream, for example, which is an IO stream that you use a node?

[00:08:15]
Obviously, IO streams can end, right? So we need some well-defined way for a producer telling a consumer, I've got no more data for you, right? So because we missed this standard way of these these three semantics, there's no well-defined way of saying completion or error. Today on the web, we have a proliferation of different stream APIs, and all of them are slightly different.

[00:08:41]
Some of them might have a way of indicating completion but not an error. Some of them have no way of indicating completion or an error, or they have a some ad-hoc mechanism where I'll send you an object, but then if you check this property etc, then you know it's done.

[00:08:53]
We have all these different interfaces. Whereas if you look at iteration where the consumer is in control, we have one interface. It's just standardized, everybody uses it everywhere. This is a mistake. We should have one interface that we use for push streams in JavaScript, and that is something very much like at least the observable.

[00:09:11]
So if you can have an iterable, so an example of an array is iterable, which is something that you can ask for an iterator from, right? So if we go back, an array isn't an iterator. It's something that you can ask to give you an iterator. Well, the opposite of an iterable, is something that you can ask to give you an iterator, is an observable.

[00:09:30]
And the way I want you guys to think about an observable is that it's a collection that arrives over time. Does that make sense? A collection that arrives over time.
>> Speaker 2: There's a question we got here.
>> Jafar Husain: Yeah.
>> Speaker 3: Can you read it?
>> Jafar Husain: How different are iterators from generators?

[00:09:58]
So the question is how different are iterators from generators? Would it be naive to say that iterators, you can get from arrays via iterator, by calling the iterator method, which we saw earlier?
>> Jafar Husain: And that iterator inherits from generators defined for an instance of array. So explaining that question will require me to explain generators and what they mean in JavaScript.

[00:10:23]
And frankly, that's a very rich and complicated topic that isn't tremendously relevant to this here. But I can answer that question, which is to say that it's fair to think of a generator as something which you can both pull information out of as well as push information into.

[00:10:37]
And the iterator that you get out of an array is just about the pull side. So for those of you in the room, you don't necessarily understand that. That's okay, you don't need to know that for this particular talk, but hopefully, I have answered that question. So for an iterator when you get out of an array, all you can do is pull.

[00:10:52]
You could you could push information back in by passing a parameter to next, but the array is gonna ignore that information. So generator is this object, this weird little object we'll talk about at some point in JavaScript, which is you can pull information out of but also send information back into.

[00:11:05]
No particular relevance, and I don't want you to think about it too hard.