Running Tasks Based on Ref

[00:00:00]
>> Mike: I'm going to go into models and I just want to add a trivial change somewhere like here. What is this? Nope, that's in enum. That's actually going to mess things up. We've got human toxicity, pet toxicity, livestock toxicity. Great. We're gonna have, Alien toxicity with affected parts.

[00:00:36]
We're worried about how our vegetables will affect aliens. All right, so I've saved this and you can see this yellow x, I've got a code alteration that's been made. So what I can do is say pnpm lerna run test --since=course-progress. What it's done is it can identify that I have made.

[00:01:13]
Gosh, there we go. That's more satisfying to look at. Well, I've touched something in the models folder and I've made a change and it looks at the delta between my current working state and any git ref here. So this could be a sha. I could have pointed to origin/course-progress, whatever you wanna do.

[00:01:43]
This allows you to run tasks on a subset of not only what you just touched, but what is downstream in the dependency graph from what you just touched. So it's a lot more sophisticated than if you've ever used something like lint staged before, where that's simply just like looking at git diff.

[00:02:04]
Which files did you touch? Let me run things based on those files here. We're getting something very important back that we had at the beginning of the class when this was a monolith in a single repo. And that was we could make a low level change to our models and then the entire test suite ran on the server, on the ui, the tests that run on the models.

[00:02:31]
And it's very easy when you start separating things out into a monorepo if you don't have the right tooling in place. You can fall into this trap where it's almost like the unit test trap, where, gosh, it seemed fine, we made this code change and the tests in that package passed, or the linting on that package passed, or this small part of the project type checked.

[00:02:55]
This lets you get back to that point where you're saying, all right, I have the benefits of a monorepo, meaning I'm not boiling the ocean and saying run all the tests over again for everything that exists in this git, git, repo. But we are getting the benefit of saying I touched something.

[00:03:12]
Use your knowledge of,where was the thing I touched? What are the downstream dependencies that it could affect? And now we can end up running a subset of the test there. So let's see if we can show an example of this actually working. What I should do now is revert this change.

[00:03:38]
Let me touch something just in server, and the hope is this just runs. So there we go, trivial change. Just the server. Wait, am I in the server folder? Could have fooled me. There you go. That's all that's running. The only thing that changed since my head commit was files and server.

[00:04:08]
So we don't need the models to run again. We don't need that test suite to pass. That's upstream. Presumably those tests pass in master. I check things out and I haven't touched it, it's validated already. So this is a really cool way, a cool benefit, a way to reap this benefit of being able to operate on test on type, check on lint on small portions of your project.

[00:04:34]
And this is where we're starting to see some of the lightweight promise of working in a monorepo come to life. But you need a build tool that's aware not just of file structures like file locations in a directory, but the dependency graph of how all of these things are related to each other.

[00:04:55]
>> Male: When we're doing learn run test, is that our old like test script we had that it's running?
>> Mike: Yep, this is the test script.
>> Male: And so how is it able to break up down to individual tests if it's just running a script?
>> Mike: Sorry, to be clear, and I'm gonna prove this, it is not in our root package JSON like I'm renaming that.

[00:05:25]
This should still run. This is going into each individual package and running PNPM test, but it's making a determination of which packages it does that in based on what was changed and its knowledge of the dependency graph. So this is very much the for each of testing. And so what I would do here is I'd say, great, we've got like the test thing in CI.

[00:05:50]
Yeah, we're going to run that. But we can also have like test changed. And this is where we would say, look, PNPM is not quite specific or sophisticated enough to do this. And so I'm going to grab this and I'm going to put it up. Here. And we can just have this be like this is what you would periodically be running over and over again.

[00:06:15]
I made some changes. Lint it for me, lint only the stuff that's changed, build only the packages that need to be rebuilt as a result of the code I have not committed yet. It's super powerful to be able to point to any Git ref. Like, you want to look at is this an incremental change that is being added on top of an existing priority.

[00:06:39]
And you know that build already passed. And so you can just look at the last three commits that you've made. So let's just do it since Origin, PR branch or whatever that is.
>> Male: So really the benefit with this is going to kind of compound as you have more and more packages, I guess in a monorepo.

[00:06:59]
Right, if you just had one package, this wouldn't really do anything. Is that correct?
>> Mike: Yeah, it's almost like the more this benefits you, the more you're able to break a sizable project up that already took a long time to build, to lend to test, the more you're able to break that up.

[00:07:21]
This approach is what lets you pay the cost, the computational cost of performing these build, lint and test tasks on the increment of what you touched and what could be affected by what you touched as opposed to the whole thing. So it pays more dividends. Like if you wanna think of it as the relative difference between those, this is more valuable as your project gets larger and as productivity starts to look worse and worse.

[00:07:51]
When you look at like what would the cost be of running the entire build from scratch over and over again? Like bluntly, I mean, this took 762 milliseconds. Let's see, what does the whole thing take? I mean it's not zero. Like this was 372. This was 1.14 11.

[00:08:13]
Like it's not the end of the world. But let's try build.
>> Mike: Lerna run build --since. Pnpm lerna run build. Great. So that was 726. But what if we just said PNPM build. All right, but remember, these are in sequence. So we've got half a second here and then another 0.7.

[00:08:48]
So we're over a second. Like this is. Call it close to double.
>> Male: It's almost double.
>> Mike: Yeah, close to double. And this for a teeny tiny little project. And mainly it's coming from the fact that I avoided building like spinning up vite to build this at all, which by the way is an incredibly fast tool.

[00:09:07]
Like you're lucky if you're using PNPM and vite and this latest version of Learna, we're leaning towards like the fastest possible things available that you could be using to build. And so, where lerna really is useful. Like say you're using regular NPM and you've got a lot more files on disk as a result and you're not using Turbo or you're not using.

[00:09:32]
Well, in this case, you'd be using nx, right? So it can add up, and it gets better with where we're going next.