Scaling Architecture on AWS

[00:00:00]
>> Erik Reinert: So here's what we're building. As you could tell, the workflow has kind of slowly grown more and more as we've moved. Can anyone tell me what is different that stands out to you?
>> Speaker 2: Got a load balancer. Cdn.
>> Erik Reinert: Yep, CDN, yep.
>> Speaker 2: RDS?
>> Erik Reinert: Yep, RDS, yep.

[00:00:21]
>> Speaker 2: And, well, ECS instead of App Runner.
>> Erik Reinert: Yep, exactly, yeah. So, yeah, we've kind of gutted most of the pipeline and changed it. But you'll notice that we don't necessarily always change everything that's purposeful. We don't want to change everything at once because then that just massively disrupts productivity.

[00:00:46]
But what you might notice, based off of our phases and what we've done, we've basically done first phase infrastructure, second phase developer, third phase infrastructure, right? And so we're able to do these pivots where it's kind of like roadwork in the middle of the night. We're kind of pivoting around the developers as they're working so that everybody can still be productive, right?

[00:01:12]
And so on the right, we still have GitHub, we still have GitHub Actions, we even still have the whole deployment process. That's a big part of why we separate these things. Because when we separate these things, we can work over here and be a more seamless experience to the developers while the developers keep working, shipping code and stuff like that.

[00:01:32]
So in this scenario, for example, what we could do is all of this work here could be done in a branch while developers are still shipping to App Runner. Then the moment when we're ready to ship over to ecs, we just merged that branch and now we're shipping to ECS instead.

[00:01:48]
So it's a much more seamless experience as well. So being aware of where you're making these changes is definitely helpful again, to the impact of the productivity of everyone else around you. Another thing to note is just that from a client and a developer perspective, this should be seamless.

[00:02:07]
So I should be able to move the entire platform from App Runner to ECS without having any downtime, having any concerns about downtime or anything like that. It's just something to note that these changes we should be able to make in a way that doesn't impact anybody and we can make them efficiently.

[00:02:28]
But yeah, we have added a few things. The first thing we added is a cdn. So we are gonna be using Cloudfront as our main CDN. Again, we're trying to stay on Amazon as much as possible possible because that's where we're probably going to get the biggest wins.

[00:02:44]
And so Cloudfront is a pretty good solution. Just a normal cdn, it's got distribution all over the world, so it helps really well with like normal static assets and things like that. What's also really nice about it is because it's Amazon, it has deep integrations with all of its other services.

[00:03:00]
And so one of the reasons why we're doing a CDN is because we don't want to make our cluster directly accessible. Like, we're trying to make it in a way where it's also like future proof. We don't want anyone to be able to like DDoS, our load balancer or anything like that, right?

[00:03:18]
And so what we're actually able to do is we're able to create a AWS ALB and create that in a private subnet. And so we can use our CDN as our front end to deliver all of our main content and APIs and everything, but then have a private load balancer internally if we want to, I don't know, pull, put a VPN in place and then make it so that developers can use the load balancer locally, but then go through the CDN for production.

[00:03:49]
Or if we want other services to communicate internally, they can communicate via that load balancer, things like that. And so, yeah, the nice thing about it all being on Amazon is that Cloudfront can point directly to that private load balancer because they're in accessible networks and stuff like that.

[00:04:07]
Once we have that load balancer set up, we are going to set up ECS as well, right? And so ECs, how many of you have used ECS before or know about it? Okay, got a couple. Okay, so CloudFront being the CDN, AWS, ALB being the load balancer, ECS is the container Orchestrator.

[00:04:28]
That's where basically all the containers are ranked. It is. I don't know if I would say it's a Kubernetes competitor. I would say Kubernetes is much better than it, but it does a really good job at running containers. If you just wanna run containers, get them up in the cloud and you want a little bit more customization around the container, basically you want more than App Runner, ECS is probably going to be your final destination.

[00:04:58]
You can go EKS if you want, which is elastic, Kubernetes, clusters or something like that, but then you're in a whole different world. So if you don't, Basically, I would say the steps are if you just want to get your app out really quick, you don't Care about anything.

[00:05:15]
App Runner's probably your best solution if you're looking for more scale, more customization, things like that. ECS is probably where you'll be. But then if you want to have a unified platform that can move anywhere and blah, blah, blah, that's when Kubernetes will come into play. So that would be like my scaling of where you kinda would go.

[00:05:37]
We're purposely not going to Kubernetes. You don't need it, man. You just don't need it. Most of the times, you just don't need it. My company runs on ecs. We do a fantastic job of running on it. I'm not saying that my company couldn't benefit from it, but I am saying that there's a lot of effort that takes to get there and you have to make that cost value.

[00:06:04]
Is it worth doing that? Is it not? And that's totally up to your organization. But most of the time, ECS or something like it can take care of it pretty well. The other thing we did change, as you might have noticed, is we also did add RDs. And you might be wondering, well, okay, we had Supabase before, why rds?

[00:06:26]
Now it's that same thing. Now we want to be able to customize, manage the database a little bit more directly. We want to have more control over it. And another big thing is we want that database to be private. We want it to be completely isolated. And because we are now fully on Amazon and in that world, we can use what's called a VPC or a virtual private Cloud.

[00:06:48]
At the end of the day, if you were wondering, what does that actually mean, it just means a private network in Amazon that you have access to, you can provision your instances to, and you have complete control over. So this really does become a lot about ownership. I want to start owning the resources that I run and I don't want others to be maintaining them for me.

[00:07:13]
Just as a kind of a comparison to stage one or phase one, you could see it's a lot more ownership, [LAUGH] right? It's a lot more. And how many people do you think it would take minimal to manage something like this? Just curious, when you look at this, how many devs do you think it would take to manage all of this infrastructure.

[00:07:36]
>> Speaker 2: You need like a DBA for sure. Probably.
>> Erik Reinert: Yeah, yeah, absolutely. I would say so. Otherwise it would be very painful.
>> Speaker 2: And then probably solid DevOps, platform engineer, whatever you want to call them.
>> Erik Reinert: Let's put it this way, if you only had one, they would be cranky, quite Often so you could have one.

[00:08:00]
I've done this as well. I was at a medium sized company where I was really the only DevOps guy. I was on a 10 man team and I hated it [LAUGH] mostly because I was the only guy. I was the only point for everything. And when you have this much complexity that is just overwhelming, it becomes very overwhelming.

[00:08:20]
And I stayed there for like three and a half, four years building, you know, everything with them. But that was a big reason why I ended up leaving was, is because I was just like, I'm the only person doing this. Like, it's just too stressful. I want to be.

[00:08:34]
There's like, there's benefits to being at a small company and then there's also benefits to being a cog in a much bigger wheel. I personally like the COG approach because there's a lot more room, there's a lot more flexibility, there's a lot more things that you can do like this, right?

[00:08:50]
And so it gives a lot more opportunity for learning, I think. Whereas when you're at a startup you'll learn a lot, but it's gonna be like through the MUD approach, you know what I mean? Whereas if you're at a larger company, you might be able to kind of like take a step back and like have a mentor or somebody else who can kind of guide you.

[00:09:08]
Yeah, yeah.
>> Speaker 2: I had a couple questions so for on rds and I guess this applies to Supabase as well. But I'm just curious like, is there any risks as far as like the developers running their container locally with postgres and then it being like slightly different in the postgres instance, behaving slightly different in Supabase or RDS and things aren't perfectly one to one.

[00:09:42]
>> Erik Reinert: Yeah.
>> Speaker 2: Is that a thing, or?
>> Erik Reinert: So absolutely, yeah, 100%, yeah. That's a great question. So if you are in a scenario like that where database parameters are very important, right, which normally they are, right? You would more than likely want to run your own RDS for starters, right?

[00:10:03]
Mostly because-
>> Speaker 2: Like as a developer.
>> Erik Reinert: Sorry.
>> Speaker 2: Like you would wanna run.
>> Erik Reinert: No, no, no, no, sorry, I mean as an organization, like if you were gonna be deploying stuff into the cloud, you'd probably wanna pick more of an RDS than a Supabase. RDS has direct access to your parameter groups and every option that postgres directly gives you.

[00:10:22]
So for example, if locally, I don't know, you needed to increase your shared connection count or some other weird obscure postgres setting, you might get that with supabase, but there is a solid chance that they're not going to expose those direct settings for you. But if you go the RDS route, you can match your local developer settings or container settings one to one to your deployment settings.

[00:10:51]
And that is actually what we do at my job as well. There are a few settings that we need because of our indexes and how much data we load into dev. Even we have a parameter group in Amazon that when the service starts, it sets those settings so that they match one to one.

[00:11:09]
The other side of it is, okay, well then how do you make sure that all the developers are in sync with the database? I actually talk about this in my first course, which is there are ways of solving that with automation. What we do and what I've adapted to now because of what I do at my job, is we use templates for all the repositories.

[00:11:34]
So, for example, the Docker compose file isn't something the developer creates, it's something DevOps gives them and says, here's your Docker compose file, it's in your repo. And then we own that file, like in GitHub, under code owners, we own all of those files. And so at any time we can go in and change them, update them, make sure they're in sync.

[00:11:54]
And so that's a part of scale is like being able to work around the people while they're working, right? So everyone in my company kind of expects anybody in DevOps to go into their repo at any time and make a change and just be like, hey, the expectation from us is then to make it in a way where it's not impactful.

[00:12:19]
We go into the repo, we make changes, we double check it, triple check it, make sure everything's working. And then we go, okay, we've deployed this new postgres version. You guys all have PRs now open. Please merge them in when you can. And then now we've all moved to either a new parameter or a new version.

[00:12:36]
We're actually in the process right now of doing that with node, we're on 18 and we want to get to 22, I think. Yeah, and so we're gonna be doing that. We're going to basically be generating. We don't manage the package JSON, but we manage the dockerfile. So we upgrade all the versions of Node in the Docker files and then we say, okay, all of your Docker files have been updated.

[00:12:57]
Please merge them in. Pipelines are green. If not, please fix and then merge in so it's a bit of a dance. Just to get the technology is one thing, but then figuring out how to have that dance of like, okay, let's make sure everybody's in sync is a completely separate challenge than that.

[00:13:15]
So, from the tech side, RDS would probably give you the ability to do that. Basically, we're going to be replacing Supabase, we're going to be using rds, we're going to be keeping ecr. Again, we have no need to move that. And what's also nice about this is we would not impact our current deployments to App Runner, even because we're not getting.

[00:13:37]
Even though we're using rds, it doesn't mean we have to delete the post or the Supabase instance right away. There could be a cost there where it's just like, okay, we'll keep Supabase up until everything's ready to go and then we'll just switch over, right? That means that deployments can keep running, and I highly encourage that.

[00:13:56]
This phase and going into this phase, and the reason why I'm talking so much is just because I want you guys to understand that you can go through this phase without making everybody mad at you. I've done it, I've gone through it, but it takes a lot of planning and consideration of what everything that you're touching and what are those points to the developers and how it could impact them and stuff like that.

[00:14:22]
So, yeah, in this case, technically, it does mean everything from phase two could stay online. You'd be fully able to keep doing that, and then you could just basically flip a switch and then you'd go over to deploying directly to ECS and your database, all that stuff. The only thing I would say you may have is you would have to do a data migration, right?

[00:14:43]
You'd have to figure out a way to get all of your data out of Supabase and into rds. And you might need like, one night of maintenance where it's low traffic and you can do that, but outside of that, it should be a completely online, seamless migration.