Developer Service Components

[00:00:00]
>> Okay, so that is the ECI or that is the Terraform cloud components, right? And as I said before, these are reusable, if it's provider, Amazon cluster, Google cluster, whatever, that's up to you. But the main point is to think of these as logical components. So that's the first thing we're gonna be building.

[00:00:17]
The second thing we're gonna be building is the service part. This is the developer side. So this side, right? This was the infrastructure side, right? The first team or set of section we were trying to solve. The next one is for developers. Now you'll see that there's components here that we create, and then there's components that developers touch that we've created for them.

[00:00:48]
Now, if I'm a developer, on the left, you'll see that if I go up, I directly go into service management, right? Terraform TFE, right? There's a reason for that, and I'll explain it further. This is kinda showing a little bit of spoilers as well. Which is we can actually make it so that Terraform TFE can even manage these configurations, but I'll show you guys that later.

[00:01:11]
The TLDR is that once we get to GitHub workspace, what do we get out of the GitHub workspace? Our repository, right? What does that give the developer the ability to make code? We just solved the problem. Awesome. Let's solve another problem. Well, the developer can make code now but they don't know how to test their changes cool BAM GitHub actions, right?

[00:01:35]
Now we have the ability to have CI run tests, make sure everything's running fine cool, okay? Now we wanted to play. Guess what, we can actually use GitHub actions for this and a separate workflow. And so in this case, we'll also use GitHub actions to go out to ECS and deploy our image.

[00:01:53]
However, we wanna empower the developers with more than just deploying images. We also wanna empower the developers to be able to update their values for their service, right? And so in this design, what we actually do is we include the configuration, non secrets, of course, in the repository with the service.

[00:02:16]
Can anyone guess why we might do that?
>> You don't wanna commit secrets.
>> Well, that part, yeah. But why would we have the actual non-secret configuration in the repo?
>> That's for the CICD pipeline to function properly?
>> Kind of, part of that, yeah.
>> We want the developer to have access to?

[00:02:37]
>> Yes. We want the developers to manage their deployments of their configurations not us, right? So this could also be another service that pulls those secrets in that the developers go and manage right like vault, right? But the main point here is that we're giving the developers the ability to configure and change the configurations of their services, right?

[00:02:58]
We're not taking care of that and we're building that into the system so that they can do that themselves. Remember, we said that the product service workspace is what deploys our infrastructure to Amazon Cloud, right? So you could see how all of these components really start to work together.

[00:03:14]
We've got the Terraform product service repository, so that if I was an infrastructure, I could come here, make changes, and update every service that's been deployed, right? Same thing with GitHub. If I wanted to update my specific repository, I could go to my workspace and update it. Every part in this stack is automated.

[00:03:36]
And it's thought out to be such so that we save time, right? Save time. Do not do the same thing twice. It's like a motto I try and live by. There's no need for you to do the same thing twice. So just automate it. And so again, you'll see that with Terraform product service, we have a repo for it, right?

[00:04:01]
For Terraform product service, we have a repo for it, right? For product service, we have a repo for it. That GitOps model of the source of truth is the repository. And it also makes it so that then the developers use these as interfaces, all right? Developer wants to make changes, they go to the repo, right?

[00:04:22]
Things like that. Any questions?
>> Not a specific question but there's a little debate in the chat about the need for this level of complexity managing this many different Nepos. Can you speak to the scenarios-
>> Yeah. So I get this a lot. Why do you need the complexity?

[00:04:48]
Let me preface this by saying this is for teams normally of at least more than one person. [LAUGH] If you are an individual by yourself, you do not have the responsibility to Rrespond to anyone but yourself. So in that case, if you wanna have lower levels of abstraction and things like that, you totally can't.

[00:05:12]
What I'm showing you is a circumstance where every one of these project repos is actually owned by a team. The DevOps teams owns all the infrastructure repos. Not everyone touches those repos, right? The developers touch the product, service repos, right? We, I don't even really touch those. Those layers of abstraction help those separations.

[00:05:39]
But if you don't have that much separation, then yes, it might make sense for you to not do that. My only caveat, and to the person in chat who said that, I want you to listen to me very carefully, okay? You still need to make sure that you are in charge of your life cycles.

[00:05:58]
If you merge your network components with your cluster components, then both are gonna get updated at the same time. You might not want that. You might not wanna make a network change and be terrified that you're accidentally gonna touch a cluster or something. So that's my only discussion or my only retort to that really is abstract with purpose, right?

[00:06:22]
If you don't need it, then it's not necessarily required. But There's reasons why we separate these. Like I said, if I wanna go to the network repository and just update every network and not be worried about touching clusters, that abstraction is good, right? It gives me the ability to modify 50 networks at once and never once in my mind have to think about any other resource, right?

[00:06:48]
And so when we talk about production changes, when we talk about really critical mission critical changes, this is helpful, that abstraction is super helpful. Reality of it is if you join stuff together, it just gives more possibility for more things to break [LAUGH]. And that's just automation. Yeah.

[00:07:07]
Yes. There any kind of sandbox environment where we can safely simulate creating resources without actually creating them for educational purposes or.
>> That's a good question.
>> Zombie AWS services.
>> Yeah, so there is something out there called LocalStack and I didn't include it. But this is actually a really great tool that we do use it hippo as well.

[00:07:30]
Now, it doesn't do everything to be fair cuz you can't do everything. You can't recreate the lambda infrastructure on your computer that would be very difficult or S3, right? And so local stack Kind of tries to meet you in the middle where it does that like mocking idea that I mentioned before.

[00:07:49]
Where local stack will give you access to the API's and other things like that. And then it will simulate that you're creating these resources so that you can run automation against it and test it. t's really under the hood just mocked API's from Amazon but it does let it so that you could run.

[00:08:07]
Like for example going back to end to end testing this is how you could potentially do end to end testing with TerraForm. Does that make sense? Because you can use LocalStack to emulate it being your cloud and then run Terraform against local stack, right? In CI, to make sure that, it won't be perfect, of course, because it's not the actual infrastructure.

[00:08:33]
But it will at least make it so that you can you can actually run it against. Temporary accounts so to speak. Yeah. So yeah, this is a good solution. I didn't include it because it used to be completely open source and then they realized that they could make money off of it.

[00:08:51]
[LAUGH] So they've got like pricing models. To give you an example, the free one basically gives you the very simple APIs, like the stuff you would use a lot, S3, EC2, stuff like that. These ones are for lambdas. You can see all the stuff in here. They can do a lot.

[00:09:12]
Certificate Manager, Amplify, API Gateway. It can emulate a lot of what you create in Amazon. Yeah. But it's a paid for product now. The workshop materials. So this is at the end of the slides. So if you wanna grab the links there you can, but these repos here are all public and they are all the repositories that we will be creating and we're gonna start working on and it basically goes from top to bottom.

[00:09:43]
We're gonna start with Terraform cloud, automating Terraform cloud, we're then gonna work on GitHub, creating our repositories. That's the next problem we have to solve. Then after that, we're gonna create our first Amazon network, right? We're gonna create a repo for that network. Yeah.
>> Are these links gonna represent the end product at the end of the list?

[00:10:01]
>> Yeah, they do. Yeah, they represent the end products. But there are branches in them, to where you can see the different progress that we made through them. So just click on the branches in the repos, and then you'll see specific sections that we're gonna be going through.

[00:10:17]
The latest is the absolute end product, but there are branches in there to kind of help you see how we change stuff and I'll give you an idea of how all this evolved.