Transcript from the "Iterators and Observers" Lesson
>> Jafar Husain: What's the difference between an array and an event? This was a question asked to me actually five years ago when I was at Microsoft when this technology was just being developed, and the answer absolutely blew my mind. It totally changed the way in which I approached asynchronous programming, and I hope that you guys feel the same way after this presentation in this class.
[00:00:36] Why do we program slurping data out of arrays differently than we would events? We should approach them the same way. How do we get to this place? How do we get to thinking about these things so, so differently when I just showed you that we can think about them in a very similar way?
[00:00:51] Well, to answer that question, you got to go all the way 20 years back, went 20 years back to 1994. It all comes from a little mistake that was made almost 20 years ago. Who's got this book or read this book? One guy back there. This was a popular book.
[00:01:06] It was a book, the idea was of 20 years, ago you had a lot of software engineers, and they got together, and they said, well, people are solving the same problems again and again, all through the industry, but they're solving it slightly differently. What we need to take these common patterns in software and put them in a book, and that's how we got the Design Patterns book from the gang of four authors there, those four authors.
[00:01:42] Now, there was a picture at the back of this book. Don't worry, we're not teaching all this stuff today. I'm actually just gonna focus on two of these things. They are actually showing how all these patterns were interrelated. But today, we're only gonna focus on two of them, the iterator and observer pattern.
[00:02:11] You see the observer way out there, but I'm gonna suggest to you, they are related. They're actually deeply related, and understanding their relationship is key to becoming an expert in asynchronous programming. So here's how an iterator works, it's very simple. You have a producer and a consumer. And the consumer requests information, one at a time, from the producer until one of two things happens.
[00:02:52] So think of this is a way of a function returning multiple values. So, an iterator object has a very simple interface. You can call next, and you'll get out this little object, which gives you the value that you requested and also gives you a little boolean indicating whether or not there are more values for you to get.
[00:03:08] So here, I got one, but I see that we're not done, so I'm gonna call next again, done now, I have 2. And I see we're still not done, so I'm gonna call next again one final time, 3, not done, call next. Finally, I get a little message that says, you know what, we're done.
[00:03:25] So go back to that consumer-producer relationship. I'm the consumer, in this case, and I'm just pulling values out of this producer, which is the iterator. Pulling values out until the producer says, I'm done, no more information for you. So the gang of four to find this pattern in the 80s.
[00:04:01] Does that make sense? That's why they defined this particular design pattern, so that you could walk up to any particular data structure. You don't know anything about its internal representation, but you can still consume the elements inside. And this is actually the underlying technology that enables the new for-of loop.
[00:04:29] Let's just think about that for a moment, right? I can write a map function, which given an iterator returns another Iterator. And every single time it pulls a value out, it applies that map function to the value and then returns the translated value. So you can apply a Map and a Filter and a ConcatAll.
[00:04:45] ConcatAll would be a two-dimensional iterator, you'd flatten it into one dimensional iterator. All the operations I've shown you thus far are actually at their root. Probably, they can be expressed over iterators. Now, I'm gonna show you another pattern that the design pattern guys expressed, and this was all about you wise.
[00:05:01] They were concerned about how is it in the user interface, if you have a change to a model. How do you communicate to the view that something is changed or vice versa? How does a view communicate to the model that a user wants to do something? And they define this pattern called the observer pattern.
[00:05:35] That's how an event works. You hand me a callback, and then when I wanna send you information, I'm the producer, I push you information one item at a time. Does that make sense? So when one of these patterns in the iterator pattern, the consumer is in control, the consumer is pulling values out of, it decides when it gets the next value.
[00:05:53] But in the observer pattern, the producer is in control. The producer decides when you get the next value, cuz it calls your callback and pushes it at you. Does that makes sense? So it's really, these two patterns are about doing the same thing, fundamentally. They're about a producer getting a consumer items one at a time.
[00:06:09] The only difference is who's in control. But the design patterns guys made a little mistake. So this shouldn't blow anybody's mind. This is just what happens, right? I move my mouse around, and we get the results. They did a little mistake.
>> Jafar Husain: They failed to see this connection, that at some deep level, these things were both about the same thing.
[00:06:29] We're both progressive about progressively sending information to a consumer, and they didn't see that the iterator and observer pattern were symmetrical. And so this is how you understand the observer pattern, if you already understand the iterator pattern, right? The observer pattern iterates you, that's what's happening. At the observer, the producer is iterating you, right?
[00:06:46] So they missed this symmetry, and as a result 20 years later, we think about asynchronous programming very differently than we probably should. There is a subtle difference between the iterator and the observer pattern as they outlined it in design patterns. Remember what I said about the iterator pattern.
[00:07:07] You can pull values out until one of two things happens. What are those two things? Can anybody tell me?
>> Speaker 2: Error and then array.
>> Jafar Husain: Error, and it's done, right? No more data, or the producer throws. That means if I call next, they could throw, right? And that's how I would know that an error occurred.
[00:07:23] But with the observer pattern, they failed to add any of those in any well-defined way of the producer expressing to the consumer that, A, there was no more data, And B, that an error occurred. Let's go back to our dawn of that listener example, right? You pass one callback to this thing.
[00:07:41] The only thing that you can receive, the only message you can receive from the producer is here's some more data. And why did they do that? Well, they were very focused on the event used case, they were very focused on solving for UI events. And the thing about the UI events is there might always be another one, right, because somebody could always click another mouse click, they could always move the mouse again.
[00:08:15] Obviously, IO streams can end, right? So we need some well-defined way for a producer telling a consumer, I've got no more data for you, right? So because we missed this standard way of these these three semantics, there's no well-defined way of saying completion or error. Today on the web, we have a proliferation of different stream APIs, and all of them are slightly different.
[00:08:41] Some of them might have a way of indicating completion but not an error. Some of them have no way of indicating completion or an error, or they have a some ad-hoc mechanism where I'll send you an object, but then if you check this property etc, then you know it's done.
[00:09:11] So if you can have an iterable, so an example of an array is iterable, which is something that you can ask for an iterator from, right? So if we go back, an array isn't an iterator. It's something that you can ask to give you an iterator. Well, the opposite of an iterable, is something that you can ask to give you an iterator, is an observable.
[00:09:30] And the way I want you guys to think about an observable is that it's a collection that arrives over time. Does that make sense? A collection that arrives over time.
>> Speaker 2: There's a question we got here.
>> Jafar Husain: Yeah.
>> Speaker 3: Can you read it?
>> Jafar Husain: How different are iterators from generators?
[00:09:58] So the question is how different are iterators from generators? Would it be naive to say that iterators, you can get from arrays via iterator, by calling the iterator method, which we saw earlier?
[00:10:23] And frankly, that's a very rich and complicated topic that isn't tremendously relevant to this here. But I can answer that question, which is to say that it's fair to think of a generator as something which you can both pull information out of as well as push information into.
[00:10:37] And the iterator that you get out of an array is just about the pull side. So for those of you in the room, you don't necessarily understand that. That's okay, you don't need to know that for this particular talk, but hopefully, I have answered that question. So for an iterator when you get out of an array, all you can do is pull.
[00:11:05] No particular relevance, and I don't want you to think about it too hard.