GitHub Actions Example

[00:00:00]
>> Steve Kinney: So, as we'll point out later, there is and will be an entire course diving all the way deep into GitHub Actions. Again, I cannot assume that everyone is using GitHub Actions. My job is to teach you some of the principles of the things that should be generally applicable, but we gotta look at something, just like we looked at React before. So we'll use GitHub Actions as our framework because 6 out of 7 or 5 out of 6 people raised their hand, but I know that there is a wide range, and there was a half hand.

[00:00:36]
Evan had a half hand. It's like a yes and hand, but the principles of all of this should be generally equitable. So we're going to look at a bunch of GitHub workflows, and I have a long write-up if you've never done one before and you want to set one up, like in terms of this repo, that is again an extension that I welcome you to do in this case, but I think for me right now, because I also want to talk a little bit about how do you test things you don't control, and a little bit about a few other topics.

[00:01:08]
So on that, I'm going to kind of talk about the high level pieces with lots of opportunities for you to dig in at a later date. The interesting part here is like, and I'm going to look at some that are from the example repo. I also just pulled up some real world ones from an open source library I used to work on, and some of my own, so we can kind of just see some in the wild examples of each. But again, I care about for our conversation right now about the patterns that matter specifically to if you have some kind of modular and/or federated and/or not a monolith, how about that?

[00:01:52]
Not anything where there is more than one thing that you are trying to coordinate, and these are about, again, the principles to apply here, which is failure mode number one, doing stuff sequentially, right? And so like in GitHub Actions, and again like these apply generally speaking, you know, there's a concept of a matrix, what I mentioned earlier, like, you usually see the examples being for building for different platforms, like if you're building an Electron app, you might do it in GitHub Actions and build for Windows, Linux, what have you, for our purposes for the app that we've been kind of toying around with the entire time, you know, we have like to say like a matrix is effective, it looks like an array.

[00:02:40]
It effectively is an array, right? Then the interesting part is like you could have more than one thing here. Let's say you wanted to, let's say not only were these each, you know, like micro frontends, but for some reason you were also then compiling them down for Windows, Linux, or whatever. Right, so the matrix could have, you know, a multi-dimensional array effectively. I have one dimension to my multi-dimensional array, which just makes it an array again.

[00:03:06]
But the interesting part is that if you find yourself repeating the steps, like, obviously using the matrix strategy is going to work a lot better because I can just say like, hey, I've got these 4 packages in this case, right, and what we're going to do is effectively like 4 is a, it basically does a for each loop, right? Because you look down here, we've got the name is going to be test matrix.package, also known as matrix.package, right?

[00:03:40]
And so it will go ahead, it will check out the, that given, you know, at this point, the entire repo effectively, right? Set up PNPM. Again, if you use npm or you need to install Node or Bun or what have you, great, cool, do it. We will set up Node. And then we will test that package, right, and we'll kind of just, it, you can effectively use string interpolation. So this allows you to not necessarily redefine every step over and over and over again, you're like, I have these 4 packages.

[00:04:14]
Different use case could be once again, like, I need to build for the different OSs or like, we're going to deploy to staging and, you know, like production or whatever, like, for any set of things that you need to do, that is one way to kind of make sure that you don't have a ton of boilerplate to look at, right? Another one is like figuring out when not to run something, right? So for instance, do you need to run your entire end-to-end Playwright suite because you fixed the typo in the readme?

[00:04:48]
Because as I learned the hard way, one thing that you kind of don't think about when you work at a company, that you think about when you run a startup, is that GitHub charges you by the minute. Right, like those CI/CD runs, they ain't free. All right, if you're in a public repo on your personal account, I believe they are very free. But the moment you are running the company and you're the one paying the bills, you find out all the things that you thought were free that are not free.

[00:05:22]
And it's like everything, right? I got the Vercel bill now, I got the GitHub bill, I got the fly.io bill. I got a lot of, I got the AWS bill. Nothing is free. Everything is expensive. And so in this case, like, you can do stuff like, okay, like, don't even run my integration tests if, you know, if the readme changed or any one of these files changed, right? Like just completely ignore it. But then the other thing that is, I think, huge to take advantage of, and again, there's a version of this for whatever you use, is this idea of concurrency, right, like, where you kind of give it some kind of identifier in this case, like it is the name of the workflow plus the ref, which is effectively the commit in this case, right, although I think the ref might be the, I had it on a previous slide anyway, you can have concurrency, but you also have cancel in progress, right, which is my integration test, you know, it's spinning a Playwright, it's building everything, it's spinning a Playwright, it's going, it's clicking around, it's doing all of those things.

[00:06:41]
So obviously I would like to do stuff, you know, these jobs, you know, concurrently, but also, if I push up another commit, stop. I don't need to, if I pushed up another commit, I do not need to finish the 12 minute integration test run across, so, you know, do you need to do all these things? Yeah, probably, but like, how do you know you need to do it? Do you want to shake your build process every time you push up because all you want is that sweet, sweet dopamine hit of it green I can merge in move ticket to done, right?

[00:07:18]
And so like doing things of like do I have a matrix of things I can kind of run across a set of things. Can I run stuff concurrently, and do I cancel stuff because again, I get the bill when that doesn't happen. Can I not run it at all if it was like a markdown file in this case, right, and a change to the readme. The other one is like you can kind of see that like, if I'm doing a Bun install, because I'm using Bun in this project, I want to make sure there is a Bun install run, but if we are running stuff concurrently and they're running in the same like, if nothing has changed in my lock file, use the last Bun install, please.

[00:08:07]
Right, do not Bun install every time on every step of this across every one of the matrix. You need to give it some kind of hash to make sure like whether or not it should, so like, I would recommend doing like the runner.os thing, especially like, because you know, in this case it's almost always going to run on Linux, but if you were building across like macOS, Linux, and Windows, like, just because you are using Bun or Node and npm, like, there are many compiled binaries that come in a lot of your packages, and like, you're usually getting the one for your OS.

[00:08:47]
So like, if we're running on the same OS, which honestly in the GitHub Action, I almost always am, right, because like I'm not building for all of these different, you can set it to different OSs, and I take the Bun lock file or your package-lock.json or your, is it a yarn.lock? Yeah, and PNPM has another one, right? Like, if the PNPM lock is the same, and we're on the same OS, I'm going to assume all the dependencies are the same, right?

[00:09:24]
And so now we're not going to run that every time. The next one is, I happen to be using Vercel. Right, when we talked about Turbo repo, it is a Vercel joint. You do not need to use Vercel in order to use Turbo repo and get the remote caching. There are many an open source remote caching server that they link to in the docs that are, you know, like, you can host it yourself somewhere and let that be there.

[00:09:46]
I happen to use Vercel, so I don't do that. But like you can have that token and team, and like those should be then secrets also running, so now like, you'll get the caching not only from GitHub, but also from anywhere else that that build process ran or those tests ran, you can keep those caches as well along the way. So before we have the concurrency and the canceling, you can choose what branches it runs on.

[00:10:17]
Is it a push to main? Is it a pull request against main, right? The unique parts will always change, right? But there's even things of like, for some test runners, you can, I think like, Jest definitely does this where you can do like only the changed files, right? And like, there's not necessarily a right answer, right? Because like maybe you do want to run all the unit tests as part of a, let's say you have the git strategy where you have like a staging branch and then a production branch.

[00:10:45]
Maybe in the staging branch, you're just trying to run the ones that have only been the changed files, but production, maybe we run all of them, right? Like, if it's, you know, if I'm opening up one PR against another, like, development branch, maybe I don't want to run the entire browser test suite because I'm going to run that before I merge the actual feature in, right, but I kind of have one PR and then like little child PRs against that branch and you can kind of begin to put all of that in place as well and like you get a certain amount of data in here too.

[00:11:22]
I'm trying like other interesting things as we kind of go along. You can obviously, you can have scripts that you can run, like, if you have a scripts folder, you can also have some bash scripts in there as well. The other, oh, so here's an interesting one that kind of fits into what we're talking about specifically in this workshop is this is one that does not run, like we always think about CI/CD running on a pull request basis because that is, for very good reason, 99% of the cases.

[00:12:00]
I think another really interesting one is, this is one that I have running on a cron job. So you can do this in GitHub Actions, where you can say basically like, on a certain schedule, I want this to run, right? And so Rodri, you use Drizzle, you were saying, right? Yeah. So what this job does is, once a day or really evening for me, I think I put it, it's like, I think like 7 o'clock or something like that, like, it will look at the actual database in production, and it will look at my like Drizzle, like tables and setup.

[00:12:46]
And if they've drifted, if what I think that the configuration is, is not the database in production, it will go down here. It will, you can see like, it will go and it will open a GitHub issue, right, that tells me, yo, what you think, you know, what is in main, where you're describing the databases and what the database is in production, they ain't the same, right? And there's all sorts of versions you could do about this.

[00:13:16]
You could, you know, you find like, oh, we stored, you know, either as a GitHub artifact, the number of deprecated calls to this function, it went up, right? And like the ability to kind of build the system, so you're not manually checking. And like, for me, it's a GitHub issue. It could be file a Jira ticket, file a Linear ticket, stuff along those lines of figuring out how do we build a system where we can kind of stay on top of these things and keep ourselves accountable, because otherwise, it will drift.

[00:13:50]
Yes, it's one about database drift, but you know what I mean, like code coverage, right, because like, I think people think that the only way to do some of these automated CI/CD checks is like, the famous Jeff Bezos thing at Amazon where it's like, it would reject if your code coverage dropped at all, it would reject your PR. And if you are the CEO of Amazon, you can say stuff like that. But like, you've got to make sure you've got the political clout for that, otherwise if no one can get their PRs in because something just happens, like you can't test every line of code all the time, right?

[00:14:26]
And no one can get anything in, you're going to have a revolution on your hands, right? And so you've got to figure out, but maybe it is, maybe you can't do it every PR because maybe that's also slowing things down, but what you could do is once a week, if it has dropped significantly, like have it send a message to a Slack channel, have it open a GitHub issue, file a Jira, whatever, whatever works for you in your organization, right?

[00:14:44]
These are the strategies that you can kind of put together in this case to figure out what we should do. So yeah, as you can see, this one is filing a GitHub issue because like my database and what it should be are not the same, like a migration didn't run something as we work through this.