Stack vs. Heap Memory

[00:00:00]
>> Richard Feldman: Bring all that together, we've got our thing back here, we've got our struct stat metadata. We're calling fstat and the file descriptor, metadata address. We asked, what if I wanna read all the bytes? This is the answer to that question. So now that I've got my metadata populated, instead of saying char buffer bracket 100, I can say char buffer metadata.st_size.

[00:00:18]
And this will basically say, now my buffer is, instead of being hard-coded at 100, it's going to reserve, again, in the functions memory, however many bytes that file size is. But unfortunately, this can very easily lead to a segmentation fault if we don't [LAUGH] double up on that, right?

[00:00:36]
So the segmentation fault actually can happen even without considering the 100 here. So what can happen here, is essentially that, if we have too many bytes for the stack, that will result in a segfault, actually because of the guard page thing we were talking about earlier. So by default, in most operating systems and most programming languages, the stack has a fixed hard-coded size.

[00:00:58]
You can configure it, but it's like, the program starts running, you get this much stack space, and that's all you get. I'm not gonna go into why that is, it's actually a good design decision for a number of reasons. But it does mean that, if you run out of stack space, then you get an error, in this case a segmentation fault.

[00:01:13]
Because we are saying, however many bytes the file is, I want you to reserve that many bytes on the stack with this buffer. If we reserve more bytes than is available in the stack, let's say, we have some file that's like 10 gigabytes or something, and that's just the stack is just not that big [LAUGH] in the program.

[00:01:31]
Yeah, you get an error right there, right, when you're trying to do this right away. So essentially, if you look at your program's entire memory, like all the ones and zeros that are reserved for your program's memory, the first section of ones and zeros is gonna be the actual executable data itself, plus some constants like the strings and stuff.

[00:01:49]
Then after that, there's some global mutable variables, there's some space reserved for those. Then you have the stack, and then after that, you have all this other stuff, which is basically the heap. So the heap is by far the biggest part of memory. This is what malloc essentially uses.

[00:02:02]
And if you wanted to, you could solve this problem by saying, okay, I wanna call malloc of that metadata size and get back the buffer. And malloc will return you an address of this, and then you will solve that problem. But then, of course, now you need to remember to free the buffer later on.

[00:02:20]
So putting these two things together, if I wanted to, I could say, I'm not gonna worry about stack size of this thing, I'm just gonna malloc of that exact many bytes. I have plenty of space on the heap for that, I don't need to worry about overflowing the heap.

[00:02:33]
Then I get the length by calling, sorry, then I call read of metadata.st_size, read that many bytes into this buffer, just like we did before, except instead of 100, now it's just however many bytes are actually in the file. Do my length check and then print that I read this many bytes.

[00:02:49]
And then, I need to remember, just like I remember the file descriptor, not only did I close the file descriptor, but also I need to free the buffer. And these can happen in either order, it doesn't really matter if I free the buffer memory or if I close the file descriptor first.

[00:03:00]
It doesn't matter, it's just I need to remember to do the sort of bookkeeping myself manually. Now, worth noting [LAUGH] that another thing that can happen is, remember I mentioned earlier that read says, tell me the most bytes to read, and then it returns how many bytes I actually read.

[00:03:17]
Now, I think that can also happen is, yeah, it can potentially say, look, if you give me a number that's too big, I'm just not gonna do that for you, I'm not gonna read that many bytes. So I actually experimented with this and found that if I asked, say, hey, read five gigabytes out of this file in a single read call.

[00:03:34]
Nock was like, nah, nah, nah, that's too many gigabytes, dial it down a little bit, that's that's too much. So the fix for this is that you read in multiple chunks. So instead of saying, I'm just gonna pass in, just read the entire thing. What you do is, you pick a chunk size that's smaller than that, and you essentially read one chunk at a time into your buffer, and then you can sort of deal with that in chunks.

[00:03:58]
And this is sort of the best practice if you're working with this in a production setting. Now, a lot of programming languages will paper over this. Of course, they're all calling read under the hood, and sometimes they'll offer you a read chunks function. Or sometimes, they'll just be like, yeah, I'll just do the chunks thing automatically for you kind of behind the scenes, and just build up a really big buffer usually using a malloc hook equivalent on the heap, just for you.

[00:04:21]
But, there's an important implication here, so let's keep this in mind as we're progressing here. If for just plain old reading purposes we want to read in chunks, what does that imply for our allocation strategy? Well, hey, if we're reading in chunks, stack is perfectly fine at that.

[00:04:38]
Why do we need to go do this heap allocation if we're like, I need to pick a fixed size chunk to read in anyway? Why don't we just go back to this original design with just hard coding the size of the buffer since I need to read only one hard coded chunk size at a time anyway?

[00:04:52]
Why not, indeed? And this is also a very common pattern that you will see, is when people are doing what's called a buffered read, where they need to essentially read in chunks, say, read one subset of the file at a time. Which is kind of the best practice anyway, in case you have a really big file.

[00:05:08]
It's actually quite common to say, hey, I'm just gonna allocate on the stack exactly the memory that I need, because I don't need to read the entire thing at a time, I just need to know how big the file is so I know kind of when to stop.

[00:05:20]
Even better, this is gonna have better performance. And also, now there's no need to remember to call free later because we're only dealing with stack memory. We do need to grant to deal with the complexity of reading in chunks, but it turns out, we need to do that, anyway, if we wanna support actually reading very large files.

[00:05:35]
Okay, so, to summarize all the things we talked about. We started out with opening a file and then remembering to close it later, very important. Getting its size by using the Fstat, and then we talked about structs and how those get populated, and all the API considerations around that.

[00:05:50]
Talked about reading the contents of the file into an actual buffer, and how we can't do reads that are too big. So, although it might make sense theoretically, to do a malloc so we can just skip the exact size, and not worry about overflowing the stack. Actually, in practice, we end up probably wanting to do a chunked read, anyway, because the operating system will otherwise give us an error.

[00:06:09]
And then, finally, we talked about stack vs heap, and sort of the trade offs there. Stack, you don't have to worry about managing lifetimes, remember when you call free. But the heap is much, much bigger, and although there's some ergonomic benefits, there are some performance downsides. Number one, malloc and free are definitely considerably slower than using the stack.

[00:06:25]
And also there are some correctness downsides, where if you call free at the wrong time, it's not just that you might get a memory leak if you forget to do it, but also if you call it too soon, you get a use after free, which is really bad.

[00:06:36]
And if you call it more than once, you get a double free, which can be even worse. Questions about any of that before we do the exercise, yeah?
>> Student: What's the reason for the stack being faster than the heap?
>> Richard Feldman: So the reason for the stack being faster than heap is, essentially, oops, that, so all of these are essentially just ones and zeros.

[00:06:54]
So from that perspective, there's no difference. If you're doing a read from the stack or a read from the heap, it doesn't matter. If you're doing a write to the stack or write to the heap, it doesn't matter. The performance difference is actually just in the malloc and free itself.

[00:07:05]
So you can actually look at, there's different implementations of malloc, but basically, what malloc has to do is it has to basically find in all of this gigantic chunk of heap memory, what's an address that's available, and can I give that back to you? And in order to answer that question, it has to do a whole bunch of manual bookkeeping.

[00:07:20]
It could have hash maps or something like that to be like, okay, well, hang on, I've used this range and this range. So this range over here, wait, that one's open, I might call free. I have to be like, okay, go mark that as free. And then next time, I'm going to try to find you a chunk.

[00:07:31]
And then you have things where it's like, there's performance benefits if you can get them to be sort of contiguous so it has even more bookkeeping to try to put things in different buckets and stuff like that. Basically, if you were to look at what the CPU is doing when you call malloc, it's just astronomically more than what it's doing when you're just putting something on the stack.

[00:07:47]
So it's just more work, because malloc has an implementation that does a bunch of work and free even more, so. Cool, yeah. So is stat a special type of struct? Great question. So stat, no, stat is not a special type of struct. This is just the definition that whoever wrote stat they just decided to name it that.

[00:08:08]
You could pick any name you want here, you can define your own structs also. So basically, if you wanted to define your own struct, you would say struct, then the name that you want, and they chose the name stat. They could have chosen foo or apples or whatever.

[00:08:20]
And then you write all of the fields and choose the names for them. Cool, other questions?
>> Richard Feldman: Yeah.
>> Student: What do you think about memory arenas?
>> Richard Feldman: Yeah, great. Memory arenas are basically an alternative to malloc and free, where you essentially, how do I explain this concisely? So I guess it would be like, if inside of two path we passed a third argument that was basically the memory address to, it's almost like you're making a little custom stack of your own [LAUGH] that can kind of like, you can just kind of allocate some memory into.

[00:09:01]
But unlike malloc and free, with arenas, you never have to free them because you never have to free the individual allocations that you make into them. Because they basically say, okay, let me reserve a big chunk of memory here, and very time I want an allocation, I put it into that.

[00:09:17]
And then once I'm all done with the entire arena, I just am like, this arena is gone now, so just throw all that away. And then nothing inside of it, well, hopefully will leak. Arenas are kind of a deep topic, but they're pretty great for performance and also usually a lot easier to deal with in terms of lifetimes [LAUGH] than malloc and free.