Load Balancers

[00:00:00]
>> Jem Young: horizontal scaling, scaling out, adding more machines. Easier to scale up and down with traffic because you design your system to be scalable and flexible. Higher availability because again, if you have a fault and a server goes down, would you rather that be one server out of 100 or one server out of five? Different perspective on thinking about availability. But it does require orchestration. And if you're moving from vertical scaling or scaling up to horizontal scaling, you have to change the way your code is factored now because now the API you're just calling, maybe it's on a different machine, so that changes the latency of it.

[00:00:38]
And that's something you have to factor in. You have to refactor your code if you're horizontal scaling, which is generally why almost all distributed system design is going to start with horizontal scaling by default, so you don't have to make these changes. But I have this big long lecture just to say it's not free and sometimes it's just easier to add a bigger machine, a bigger database, problem solved.

[00:00:59]
You don't have to go through this whole system design thing. But I'll say in an interview, it's almost never the answer because it's too easy. No one's going to let you get away with that. So let's say we want to scale our to do app. Remember, we were just sitting at a client's server, one server in a database, it handles just fine. That can handle 1,000 users easily. We say, oh, now we want to do 100,000.

[00:01:29]
What do we do? We would scale generally out, we're going to horizontally scale, so we're going to add more servers. What's the problem with that approach? We're probably getting too far ahead of ourselves. We're thinking like, there's only three components of the system. There's a client, there's one server, there's a database, and now we're adding say three servers. Your database might bottleneck if it's not scaled.

[00:01:53]
Yeah, yes, yeah, or other downstream resources, that's true. Your bottlenecks may change now. Now you discover actually it was your database that was slow, preventing you from scaling load balancing. Thank you. Yeah, load balancing, because before your browser, your client was just talking to one server, easy. You don't have to worry about IP addresses, it's all managed. But when you have multiple servers, now you have to keep the DNS updated.

[00:02:18]
You have to know, hey, one server's offline. How does the, where does the client know to connect? And that's why it's probably challenging because we talked about load balancers earlier, but that's what a load balancer does. It makes it easy to add scale up and scale down really, really quickly. And this is a real problem that it's solved by relatively straightforward technology, but if you didn't have a load balancer, things would go wrong very quickly.

[00:02:41]
So, I'm sure there's some sort of name for that kind of solution where it's just like, oh, it's really simple, but I can't think of it right now. So a load balancer can do a lot. It can be a termination for your HTTPS. It can route based on, it can do deeper inspection of the routes you're trying to do. It can handle health checks and things like that. But usually if you get to this level of load balancing, you're actually just talking about a reverse proxy, and reverse proxies can do load balancing.

[00:03:13]
Most of them do. Nginx is, you know, tried and true, it's the one we use in full stack for frontend, but it can do all these things more capably than just a pure load balancer. But if you have a very, very high traffic application, you want a dedicated load balancer, something like, I don't know, Amazon Elastic Load Balancer or something like that, which is just a load balancer. But generally, there's a lot of flexibility between reverse proxy and a load balancer.

[00:03:41]
I won't say intertwine them, but if you're doing a system design, you just say load balancer and assume it's doing all this other work. And if they want you to specify, you can be like, oh, it's actually a reverse proxy. I was just kidding. Easy. So now we completed a high-level design, and now it's a little bit better. We're going to sketch it out in a second. And we learned about vertical scaling and horizontal scaling.

[00:04:02]
It's a trade-off. Everything's a trade-off, but generally we consider the tradeoff and managing complexity of scaling out or horizontal scaling to be worthwhile. But I still say keep vertical scaling in your back pocket if you ever need it, because sometimes it's not worth it to refactor your database and shard and partition it when you can just make a bigger database.