Adding Cluster Configuration to Environment

[00:00:00]
>> Erik Reinert: Cool. Okay, so I have my distribution, I've explained that for you guys. Let's go ahead and add the normal suspects. So we're gonna add our variables.tf, right? We said that these are pretty much in every one of the modules. Again, you can see here the capacity provider that we create, the name of the module, the security groups, the subnets and the VPC id.

[00:00:21]
This is all related to creating a cluster. But again, if I wanted to take this cluster and use it in other automation, these variables shouldn't be enough or should be enough to be able to do that. Then I'm going to do outputs TF and I'm going to add my outputs.

[00:00:37]
And as we said before, these are things that other resources can use, which more than likely means that these may be things that the services end up using. Right? So we're going to go ahead and output those. And then the last thing we need to do is we need to go to main or, I'm sorry, we need to go to the main inside of the environment directory and we're going to add our cluster configuration to our environment.

[00:01:07]
So really quickly, each one of the modules in this main TF are our own custom modules with parameters that we have set. So again, our network, our database, these are variables from those variables TF files that we said we needed. And we are doing the exact same thing with the cluster.

[00:01:30]
Now what's interesting about these configurations is if in the future you had a desire to create a cluster network and you wanted to make it so that every environment now provisioned clusters on those cluster networks, you should be able to do that. You would just add a cluster network to the main network module, you would export it as a variable and then you would put subnets module network clustersubnets instead of module network privatesubnets.

[00:02:03]
So this is why these aren't hard coded. You have the ability to play and, or change things if you need to. Again, in this case, we are wanting the cluster to be on the private network entirely. So we say module network, private security group. We want it to be on the private subnets.

[00:02:23]
So we say module network, private subnets. We give it the name. This also means that our cluster is per environment because we're not creating multiple clusters. Now what's kind of neat is every single one of these components can also be duplicated. So what do I mean by that?

[00:02:43]
Well, what if you want two networks in your environment? What if you want three databases in your environment? What if you want multiple clusters in your environment? You can Easily change this, adapt it to whatever you need. One of the things we would do at my job is we would actually create a cluster per what we call tier.

[00:03:05]
Tiers where I work are effectively the layers of applications that we create and deploy. So for example, we have a backend tier and those are all private, completely cut off from the Internet services that can be accessed via HTTP. And then we have a gateway tier. The gateway tier is the exact same thing as a back end tier, but it has access to the Internet, meaning that those services can then be requested directly.

[00:03:35]
Then we have a front end tier where all of our front end services go and then those have a load. So we have multiple clusters that further logically separate our services and what we're deploying this granularity again, still entirely up to you. I'm just giving you a template to start with and work with.

[00:03:56]
The other thing I really want to note here before I provision everything is the capacity providers. This is entirely a setting that I have created to solve the problem of I want to schedule containers A on instances B, right? So for example, if I wanted a bunch of containers to get scheduled to the spot instances or the spot capacity provider, I would tell it, hey, the instance type is, is whatever.

[00:04:25]
And then the market type is spot. Cool. Now I have capacity. And so for example, if I wanted to say like spot medium, right? And then I wanted to say like, you know, spot large. And then I wanted to say this was like a extra large, right? Now I have medium instances that are on spot and I have extra large instances that are on spot and I can provision my containers to either.

[00:04:50]
Or if I wanted to, I could say, you know, something like on demand large, and then just say on demand here, meaning now I have small spot instances that I can provision to and I have large on demand instances that I can provision to. So the point of the capacity providers is really to give you that complete customization of what kind of instances are you provisioning and where you can put those containers when they're ready to be scheduled.

[00:05:18]
For now we're just going to put everything on spot and we're going to use the smallest instance size we can for what we're trying to do. So we're going to say spot. We're going to create one capacity provider called Spot. Its instance type is going to be a T3, a medium, and the market type will be of course spot, because it's a spot instance.