Iterator Memory Usage

[00:00:00]
>> So let's do one more thing. I need you to do this in order, okay? I want you to skip every other line, then I want you to skip the first two lines, and then I want you to print two lines. So you have to do that in order.

[00:00:11]
So I'm gonna walk you through it, we'll do it together just because time is of the essence. Okay, so right here, let's skip two lines and then we need to take two lines. So I'm gonna just do another filter and I am going to do an x, and I'm gonna do an i, and I'm gonna say, while i.

[00:00:35]
Let's say i have to be greater than 1, right, because 0, 1, skip the first two. And i have to be a less than, so we need the next two coming out of this, which is gonna be 2 and 3, and so i have to be less than 4.

[00:00:51]
Does that make sense? We're just really just doing a simple little, wait, what? There we go. We're just doing a simple little skip here. Grab the first two or skip the first two, skip everything above 4. Does that make sense? Or skip everything above and equal to 4.

[00:01:08]
There we go. We could have used some equal to that might have made it a little bit more clear. But there we go. This seems pretty good. And so now when I run it, I should only see 2 and r if I'm not mistaken. That is cargo 2r.

[00:01:22]
So we've correctly filtered, skip the first two, print the next two. And so we've done that with two filter statements. So let's do the same thing, but let's do that in Rust. And if I'm not mistaken, I didn't get the exact same code cuz I think I knew I used a little equal then right there.

[00:01:43]
All right, so let's do the exact same thing in Rust except for we have a bunch more combinators, so we could probably do this a bit more intelligently. So let's jump over here. So after our filter, I can just say skip(2). Awesome. Then take(2). There we go. Skip the next two, take the next two.

[00:02:05]
Notice we're not doing some crazy index kind of guessing here that's going on, it's just skip two, take two. Fantastic. And so now if I were to run it, we get the exact same results. Again, geniuses, all of us. This feels fantastic, yeah.
>> Performance question. The TypeScript one used filter, presumably we had to go over the whole list.

[00:02:26]
>> We're gonna get to this, you're hitting on a very good point. So if you didn't hear, asked a performance question, is there a performance implication with skip and take versus doing a filter? Yes, the next section. So let's break down what happened here. So this is really what we just got done doing in TypeScript, right?

[00:02:45]
We split, we filter, we filter, we print. There's not really an easier way to do this. I guess, we could have tried to figure out a way to mash it into here, but then we'd have to say, if i equals 02, if i equals 4 and 6, also return false, but 8 and 10 don't return false, right?

[00:03:02]
It'd become a very complicated statement. I feel like this is just easier to read, I think we all agree with that, just nicer. All right, so split takes your entire string, reads it character by character until it hits a new line. When it hits a new line, boom, you have a string.

[00:03:17]
Then it keeps on going, hits a new line, boom, you have a string. Adds these one at a time into a list. Are those new strings? Are they references to the original string? I have no idea, but they exist inside of a list. Filter goes through that entire list one at a time, calling your filter function.

[00:03:35]
Anything that returns true, it will put into a new array. This is very important. Your next filter, we'll do that again, it'll go through the entire list, anything that is true, it will put into a new array. And then finally, you do a for each over it and just iterates, and hey, I approve this, great method.

[00:03:53]
Okay, for each, great method. So let's kind of think about it. If you were to write the code yourself, you'd write a function called filter_1, you'd write a function called filter_2. You would then read through a file, letting it going through every character and creating the array. You'll create a new array b, go through, put in only the true values.

[00:04:15]
You create a new array c, go through, put in the values. You would then go through c and print out every result. So that's really what our code was doing. We just made it much more condensed and easier to read, but that's what's happening. Now v8 may do some sweet optimization, I can't tell you what v8's gonna do, but it's gonna do something.

[00:04:33]
Now let's do the exact same example here in Rust. All right, so this looks pretty good. So what it's gonna probably break down into is gonna look something like this. We have a start, we have a taken, we have a skipped, we have a lines found. I am then gonna do a lines.enumerate().chars().

[00:04:50]
So in other words, I'm gonna look at every single index on every single character. All right, so here we go. Until the character is a new line, we're just gonna keep on going on this loop. Once we find it as a new line, I'll create a reference to the original string from start to that index and adjust my starts to the next starting point.

[00:05:10]
Pretty straightforward. I have now found a line, hurray. If lines found is not %2, we just go back up to the top, keep looking for a newline character. Notice we're not going through the entire string, we're only going through line by line. Rust is a pull operation, JavaScript is a push operation, it pushes you the values.

[00:05:32]
Rust, you have to pull out the values, which means that it's very lazy, it executes it one at a time. So, that means skip is the exact same thing. Until skip has been done twice, we're just gonna keep on continuing, then we'll call take or then we'll increment taken, we'll print it out in our for each.

[00:05:47]
And then if taken is equal to 2, we break and we do not continue to process. So you can imagine if this file was 10,000 lines long, we'd process 8 lines in Rust, we would process 10,000 lines, 10,000 lines, 5,000 lines, 2 lines in JavaScript. Very, very different, though the code looks effectively the same.

[00:06:06]
So, this is some of the benefits of having an iterator versus having a .map method. They're very, very different and you have to have that in your head that an iterator is a separate structure in which either refers to or owns a value and walks through the values lazily.

[00:06:22]
When you call next, it gives you the next value, it does not process anything, it just does that one atomic operation. All right, there we go. So you'll hear this phrase all the time, zero cost abstractions. People love this phrase. This is why when you learn Rust, you can go on Twitter and say, I've learned Rust.

[00:06:41]
And you get to tell everyone you've learned Rust and you know Rust, because that's what you get to do, it is one of the benefits, zero cost abstractions. We had an abstraction, which is all these iterators, but ultimately, they compiled down into just like a simple loop in code in the actual machine code.

[00:06:55]
You can go to godbold.org and look at it, it is literally just a loop. And so it gets rid of all that abstraction for you, it's beautiful. So do we have any questions?
>> Would you explain the pulling and pushing again?
>> Yeah, yeah, yeah! So if you think about it, when you do any array.map, you don't have to collect this, it does the collection for you, right?

[00:07:20]
It puts it into a new array. So when you call map, it goes over the entire list, pushes it all into array, and then hands you the final result. If you were to do this in Rust, it does nothing. You didn't do anything, it just created a couple data structures and that's it.

[00:07:36]
Because you haven't called next, you haven't pulled out painstakingly each value one at a time. So that's the difference, is that TypeScript immediately or greedily executes it. It pushes everything through, whereas Rust, you pull them through. Which means each iterator will call next on each parent iterator one at a time until you get to the source.

[00:07:57]
And then that value flows through the iterator until it successfully happens.
>> This reminds me a lot of C-Sharp and Link, I don't know if you have much familiarity with that. But they have this notion of deferred execution, so you can build up a pretty complex set of processes and it doesn't execute until you actually iterate.

[00:08:19]
>> Yep.
>> Is that -.
>> Yeah, link, I believe underneath the hood creates an iterator or they might create an observable. I'm not really sure exactly what it is, but those are kind of two very similar concepts, I guess. And if you've ever used iEX, which is like the sister to RxJS, it operates over arrays and you can create the exact same thing.

[00:08:39]
You create all these iterators to pull values through as opposed to immediately executing them. So it's like a very similar concept.
>> People are saying also RxJS and nothing happens until you subscribe or -. Exact same concept.
>> For Java streams.
>> Yeah, so it's a lot, anything that requires these kind of pull operations, everything JavaScript does generally is like a push operation.

[00:09:04]
When you create a promise, it immediately starts executing. In Rust if you ever get to async programming, when you create a promise or a future, it doesn't execute it until you call a wait. You are responsible for pulling out the value, it's just a different paradigm, but it leads to different implications in the end.

[00:09:22]
And I think for this, it's very hopefully this was extremely clear why this is so dang good, because you can really control exactly how it operates whereas in JavaScript, you have no control. You are kind of at the mercy of whatever the standard does, and that's what it does.

[00:09:36]
And so it's just it's a little bit different. And control comes with more boilerplate, but control also comes with really great performance and sometimes even better ergonomics. And I'd argue that Rust can often feel better than TypeScript.