Terraform & Pulumi

[00:00:00]
>> Automation tools, yay, next one. And I actually felt that this gift was a pretty good representation of automation tools. Because I think that this is what people kind of see quite often is they see these two paths in front of them were like, well, if I use this it'll be sunny and everything will be perfect.

[00:00:17]
But if I go the, it's, this in my mind is truly how it feels to pick automation tools. So we're going to talk about some of the industry standards first of automation tools. I'm not going to go super deep into them on a technical perspective. I'm just going to show you them and kind of give you an example of how to pick between them.

[00:00:37]
So the first one is Terraform. The star by the way means that we're going to be using it in this course. So we will be using TerraForm in this course. Pulumi also exists. Terraform is widely used. It's been around for a long time, basically 10 years. And it's just it's it's pretty much standard.

[00:00:56]
Everybody uses it. Pulumi however, is growing. The DevOps for Developers course that we did before, that actually does cover Pulumi. And if you have heard of some recent changes with Terraform, Pulumi and even other things are becoming a bit more popular now because of those, which we'll talk about in a minute.

[00:01:16]
But honestly, these two right here, in my personal opinion, can take you to scale. They can take you to where you need, it's just figuring out how to get there with you know the implementation that you need. OpenTofu, okay, I didn't want to add this but I felt like I had to because there's some stuff that happened a few weeks ago with Terraform.

[00:01:41]
And it may it was enough to disrupt the section of DevOps that we all work in, so Terraform decided to change their licensing on the Terraform CLI. They basically said you know what, we kind of want to make money off of Terraform. So we're gonna move to BSL license, which is essentially a business license, meaning that you can't use it and resell it unless, well, you just, you can't, [LAUGH] they don't let you do that anymore.

[00:02:11]
So what happened was the moment they changed their licensing, a ton of companies went into the Terraform repo, cloned it and took all of the data before the licensing, [LAUGH] all of it. And then that through that was eventually this thing called OpenTofu. Now, it was originally called OpenTf, but the main point of it was to get the knowledge and the basically everything that TerraForm did and try and keep that as open source as possible, right, so OpenTofu is basically just tariff Terraform.

[00:02:48]
It still works like Terraform. As a matter of fact, if you add a Terraform module to OpenTofu, it still says Terraform. None of the logic has changed. It's meant to be a drop-in replacement. Now, does that mean that you should worry about it? No, and here's why. For starters, it's not an industry standard.

[00:03:11]
It just got created two weeks ago. Sure, it's Terraform, but it's a brand new repository. We have no idea the direction it's gonna go in, and maybe. The caveat or the thing that I put here is that OpenTofu really only helps at the moment, right now, during, recording this, with licensing with developers or providers who want to resell Terraform-like solutions, right?

[00:03:38]
So if you're a developer who's like, man, I want to take Terraform, build a platform with it that people can run Terraform. You can't do that anymore with Terraform. It's against their policy. It's against their license. And you'll get in trouble for it. But if you use OpenTofu, you can do that.

[00:03:55]
So what does this mean? Well, this could change how things work. Right now we use TerraForm and these other tools as like local CLI tools that we run against the cloud automation to create and provision these systems. But and again, this is just my own, putting on my wizard hat for a second.

[00:04:15]
There's part of me that thinks that they're going to try integrating these things directly, so that I could just copy and paste a piece of TerraForm code and it just creates it. Right, no need for CLI, no need for any of that. So I do personally think that that's where a lot of the innovation for OpenTofu will exist now, which just means still, it's not something that's really important at the moment.

[00:04:38]
It's just something to make sure you guys are aware of and really just to kind of avoid, you don't really need to mess with it now, at least. So let's talk about Terraform, right? What is Terraform? If you've never used Terraform before, it's a declarative way of configuring your infrastructure.

[00:04:57]
Does anyone know the difference between declarative and imperative, by any chance? Chat, maybe yeah.
>> Declarative is when you have like a surface level API where you can kind of build on top of it. Whereas imperative is the actual kind of step by step
>> Yeah.
>> Implementation of that API.

[00:05:17]
>> Yeah, exactly, yeah. So you the biggest thing to take away from Terraform is it's just focused on configuring things, right? There's no heavy use of like logic, that you'll use some but really it's about copying and pasting things and configuring them. That's it, right? So Terraform has a very low entry point for starting, really anybody can get started with Terraform as long as you're comfortable with using a CLI tool and reading docs, you'll be good with Terraform.

[00:05:47]
And another part of Terraform is has it has massive developer support, right? It's been around for, again, a long time now. We've built and iterated on top of it, and multiple cloud providers are supported by it. So you can provision on Amazon, Google Azure. It's just again, it's the industry lead right now for a reason, and it's because of its adoptability.

[00:06:11]
So this is an, this is an example of what Terraform Code looks like, right? And when we break it down, it's, actually very, very simple. So the first component we have is the resource. So that is just what's called a meta argument. It just means you're telling TerraForm I'm creating a resource, right?

[00:06:29]
The next thing is is the actual resource you're creating, right? So when you think about being a developer and you want to like create infrastructure, this doesn't include, how do I set stuff up? It's just like that's all you, literally that's all you need. To create AVPC in Amazon.

[00:06:48]
Then you give it a name, right? Your resource name. Now, does that mean that that's the name of the thing that's actually created? No, that's just the Terraform state name so that it knows how to save that data. And then we have our curly brackets. And basically within those curly brackets is your actual configuration.

[00:07:06]
So, to create a resource, all you need are three separate strings, and then a brackets, right? Everything inside of those brackets at that point are going to be unique to that resource, and things like that. So in this case, we're building an AWS VPC, it's on a cider block of 10 00 16, it set to default and we're giving it some tags as well, right?

[00:07:29]
So it's 1, 2, 3, 4, 5, 6, 7 lines of code, and you have a complete virtual VPC created for you, at least. Now, is it, exactly what you want? No, but you can go in and add changes. So let's talk about Pulumi now, right? We talked about Terraform a bit.

[00:07:48]
What are the differences between the two? Well, like I mentioned earlier, Terraform is declarative. Pulumi is really imperative. And I actually, [LAUGH] I sat down and I really tried to think about this really hard cuz I think people might argue with me on this. However, Pulumi focuses on using programming languages that are imperative.

[00:08:07]
[LAUGH] So no JSS, Python, Go, .Net, Java. Basically, all of those besides YAML are imperative languages that are designed to take things through steps. Now, I just told you that Terraform is declarative, so this might already make you see the difference between the two. Where this is much more like a normal programming approach, where you have to add this, then you add this, and you write more code.

[00:08:36]
You're basically building the process out imperatively in the logic because of how the language works. Whereas Terraform, I can have this resource over here, this resource over here, this resource over here it, and it doesn't matter. It will just, [SOUND] move it all together for me. With Pulumi because it's imperative you're using a language.

[00:08:56]
So you're not using just pure configuration language in that case, which is HCL but in this case, it gives you the ability to use like node, Python, go.net, Java and again YAML. So what does this mean like why? He's like, okay, that's great. But like, whatever. I don't care about the language I use.

[00:09:15]
Well, you might, you might. A good example of this is like, security, right? Maybe TerraForm needs a way of access, or I'm sorry, maybe your automation needs a way of accessing a very unique service. Right, and It's just hard to do in Terraform. How do I make a request?

[00:09:34]
And this is something you'd think a language would be easier to do, right. And this is where Pulumi can really shine, is basically doing all the things that Terraform can't do. The HCL language that Terraform uses is good. It's very good, but when you talk about what's actually general purpose language, there's just things you can't compare it to, right?

[00:09:59]
With a general purpose language, you can do like heavy modularization, right? You can create your own repos of like templates, or libraries and then those create your resources for you. And you can do stuff like that in Terraform with modules, but my main point here is that you're doing it in code.

[00:10:17]
So all of the flexibility that you would have with a normal programming language, you'll have with this, right. And that also makes it so that it's more flexible as well. So in the future, when you have to maybe add something that's really niche or really particular, that's actually easier.

[00:10:33]
You don't have to go, how do I make this work with Terraform? You just go, okay, well, how do I program it, right. This is an example of Pulumi. So you're going to notice automatically that we already have things that we didn't have before. The first one is that we're actually importing the libraries that Pulumi needs to use because Pulumi, right now at least, we're using Node .js with this.

[00:10:59]
Because it's a language, I couldn't just go in to go and start writing automation. Because go is just a programming language it's not an automation language, so what we have to do and this is why I say you have to really consider this part, right. In this scenario before you can automate, you have to set up the repo, you have to go in, you have to set up your packages, you have to set up your package.json.

[00:11:28]
There's work that has to be done here before you can actually start moving forward, right. Whereas when we go back, it's just go, right. So this is a big difference between Paluma and TerraForm that I really want to shine here, is it's not that they both can't do the same thing.

[00:11:47]
But with this, it's strictly designed to just go. Whereas in this scenario, it's strictly designed to say, tell me how you want it pretty much. And in this scenario what we do is we import Pulumi, we import AWS, if there was Google, we'd import Google, right, we'd import other things.

[00:12:05]
And then we'd start creating our resources. Now again, they look similar. If you look at like the variables right cider block instance tenancy, cider block instance tenancy. Can anyone tell me why those are same still?
>> Cuz the API's underneath are pretty much the same?
>> Exactly, yeah, the API's for the cloud provider are the same.

[00:12:23]
And this is another thing that you can note and take away which is, whether you go to Pulumi, whether you go to Terraform, sometimes they're interoperable. Because they're all using the same cloud providers at the end of the day, so it's not as scary sometimes to say, well, let's move from Pulumi to Terraform or back and forth.

[00:12:43]
Because you can lean on that most of the time. But again, the big takeaway here is that you're using logic to build your code meaning that this is much more imperative. So, which one should I use? Which one should I use? Which is the question I get all the time?

[00:13:02]
Well, how to choose, in my opinion, comes down to three main things. Promise me you will stay true to these three things, okay? [LAUGH] The first one is focus on the problem you're trying to solve. So many times in the DevOps world, there's a new piece of tech that comes out, there's a new way of automating something, there's a new fancy.

[00:13:24]
In a lot of ways, all of us nerds are just like moths to a fly zapper, [LAUGH] like, we just see light and we go. And so don't do that, don't do that here. Use that, do that in your free time. Right, but don't do that here. Right focus on the problems you're actually trying to, like if you can make VPCs and TerraForm, stop.

[00:13:48]
[LAUGH] You're done, you don't need to go any further, right? But if you have a need for the language, like you want to work in Python or JavaScript or whatever, then Pulumi might be a good bet for you, right? Focus on how efficient you want to be. Anyone from the Pulumi company?

[00:14:06]
However, I do think that once you start using code, it becomes harder, yeah.
>> Why not use the AWS CLI directly?
>> It is a good question, which is like, why not? Like, the CLI, the Amazon CLI does provide really cool stuff. Like, for example, if you want to create a whole Kubernetes cluster, they have one command for it, why not do that, right?

[00:14:28]
I go back to the get ops answer answer. How, I'd go back to what AI said. How are you? How are you gonna do it again? How do you, how are you going to know how you did it last time, right? That's where automation comes in, right? You take get ops, you plug it into automation.

[00:14:45]
And now you have a source of truth, as well as a way to deploy that source of truth, constantly. You know what I mean? If you're doing it manually, this is you down here, and you're taking it all the way over here each time. You know what I mean?

[00:14:56]
So not only is it, do you want to save money, but do you want to save time? Like, do you want to have to do that every time? Or something or have something do it for you automated, yeah.
>> My mental model for the difference between Terraform and Pulumi is that Pulumi can be as arbitrarily specific as you want.

[00:15:14]
>> Yes, yes, yeah.
>> And the Amazon, the AWS CLI tool is like as unspecific as you want, just get the effect, some effect.
>> Pretty much, yeah. I mean, so I think maybe a way I can add to that is, when you do it via the CLI, you also have no idea when it's going to change.

[00:15:29]
You have no idea if they're going to change arguments. You have no idea if they're going to change how they do it. You have no idea if they're going to change arguments. You have no idea if they're going to change how they do it, right? So I would say like ad hoc stuff, stuff that you really just need to kind of do once, CLI is perfect.

[00:15:44]
But if it is a repeated process, that should go through some type of automation because then we can track it and stuff like that. Beware, like I said, do not focus on what is best or most popular, really unless you want to waste time. You'll waste a lot of time, and you'll normally bounce around a lot.

[00:16:07]
So I did write some cases for us to try and when is the best time to use the right one. Any one from this is just my own personal feelings, we do have people from both of that [LAUGH]. For me, Terraform was about the desire to use declarative configuration for starters.

[00:16:26]
That's the massive separation between Pulumi and Terraform. And if you just don't really care about writing it in a language that you want, I don't really think there's any value to doing it. The desire to use a single language for all configurations. That's another big one. Pulumi is really meant to be like, if you want to use Go with TypeScript and da-da-da.

[00:16:46]
If you really aren't going to use more than one language TerraForm just might be a good fit for you. Yeah, it's taking the problem and making it much simpler, right? And then desire to use a single tool for all configurations. That's kind of the same thing, is TerraForm can take care of provisioning this and take care of that.

[00:17:03]
It's kind of like an all in one solution and that's why It's become so popular, right? So the TLDR use cases is basically when you got the need for speed, when you want to move fast, if you want to keep moving TerraForm can help you do that and listen like I've ran Pulumi for a long time.

[00:17:23]
I really, really, really love it because of the flexibility and the ability to write in your own language and stuff like that. I cannot argue with the fact that Terraform, I'm still faster in every time. And again, when we go through Terraform later on, you'll see how quickly we can really do this without having to have a barrier of like language and compiler and transpiler or things like that, right?

[00:17:49]
For Pulumi, right, if you want to be imperative, if you actually have an infrastructure where you're kind of like, okay, I want to do something, and then I want to stop, I want to do other things. Pulumi will be more flexible with that, because then you can lean on the language to, throw errors or other things like that when you want to, you can't really do that with Terraform.

[00:18:08]
So Pulumi is really powerful with being able to use the imperative configuration language, I guess. Again, the desire to use multiple languages. I really think this is a bigger one. I think that if you do want to use multiple languages, Pulumi is probably gonna be the best angle to go.

[00:18:27]
Because then, going back to that team approach. You can have some people writing Pulumi over on this team, you can have some people write Pulumi over on this team and you don't have to tell both teams. You got to write it in this language. So that's really nice and the desire to use multiple tools.

[00:18:41]
So you might be like, well, wouldn't you just be using Pulumi in this case? What I mean by this is that because it's the language that's there. You can also use anything else that works with that language, right, so if you have clients, right, where they connect to like other API providers and stuff like that.

[00:19:00]
You just can't do that easily with TerraForm but because it's a language you can do that really easily with Pulumi. And yeah Pulumi is covered in depth in the DevOps for developers course. So I would recommend watching that course if you're more interested in Pulumi, too. And yeah, so the TLDR for Pulumi perfection.

[00:19:19]
I genuinely think that's what it is. I think if you're just trying to go for the absolute perfect, ideal, like everything you want across the board you made yourself, then I think Pulumi really is a great choice. Terraform is about saving time, right? Pulumi, I think, is really about, like he says, perfection, exactly what you want, the specific needs or scaling requirements.