Turborepo Builds

[00:00:00]
>> Steve Kinney: We took a tour of the repo already. I can do this pnpm -r build just kind of recursive go through and build, and it's going to build everything all the time. So I could run it, it's going to build everything. I could change one file, it's going to run everything again. I am happy to demonstrate that for you if you want. Does anyone want to watch me run a build process three times and watch it do the same thing, or do you want to believe me?

[00:00:24]
Alright, so this is effectively what the dependency graph would be, right? In this case, right, which is the dashboard is effectively that shell app. In this case, it's pulling in those analytics that bar graph we saw in the first example. The user is now its own sidebar and it's not owned by the shell. Those both rely on the UI, but then the dashboard also relies on the UI and everything relies on shared, right?

[00:00:54]
So one would argue that if one read this graph from the bottom up, right, that should be how the builds work. If I changed something in shared, well, everything should probably rebuild. But if I change something in users, probably just users and dashboard. I don't need to rebuild all of this. I don't need to run all those Playwright tests, so on and so forth, because nothing changed since the last time.

[00:01:24]
But things that, you know, like the cascade of dependencies, like the stuff that did change, you should be able to run, right? And if someone's going to ask you the question of like, well what about this edge case where I have this unique setup where things can fix it. Right, you should do the work to make sure that like, you live in a world where that kind of happens. And like I said before, as it does all those things, it creates a content hash.

[00:01:53]
If that content hash hasn't changed, right, it'll be a cache hit. It's effectively like a CDN or HTTP caching. We don't need to redo stuff that has already been done. But if it didn't and we don't have it in cache, then go ahead, yeah, run it. Okay, so we're going to install Turborepo. If you've only ever used npm, guess what add does? Give you a hint if you've ever used yarn or pnpm. I think I've always just run pnpm install.

[00:02:25]
I decided to like follow the rules. -D is as a dev dependency, and then -w means not in any given one of these packages, but actually at the workspace root. Pnpm -D -w. Gotta do the add part. You could type install there, it'll work. Turbo. Great. And so now that is installed. I guarantee you it's also in there, it's turborepo. And cool. So let's look at like what a turbo.json looks like, and there's not a lot of crazy in here.

[00:03:05]
It's pretty straightforward. The one nice part is this is optional. But if you put that in there, like that's a link to a JSON schema and which defines like what a JSON object needs to look like. You don't have to have that, you're like, that's confusing, I don't know what that is. If you put that in there IntelliSense is your friend and like you'll get like effectively like type checking on this file in case you flub something.

[00:03:35]
And if you don't put it in there, then don't mess up. You'll just get an error, it's not a big deal either way, but it's a nicety. And the nice part is for you, if you ever had like configuration files that you wanted to like provide that same thing, effectively, like VS Code and most IDEs will support this. So if you had config files, let's say you were running a whole bunch of repos and you are the architect for the, you know, the front-end architect of the company, you could have a bunch of these schema JSONs published and then like for everyone's config files, you could like literally have a check process to make sure everyone's configs were correct and give them the IntelliSense otherwise.

[00:04:15]
So there's a freebie for you. It's like not a special Turborepo thing, literally VS Code and like there's a library AJV that will also like check these files too for you and so it's nice, it's great. We'll go and we'll copy that and I'll explain it in a second. So I'm going to just make this turbo.json. You misspelled Turbo right? Whatever. Probably I just gotta reset the TypeScript server in a second, but it's fine.

[00:05:01]
Let's look at this because again, we are at this point. I'm no mathematician. I'm going to say 85% done setting up Turborepo. Right, maybe even 90, like we're a, depending on what your definition of done is, possibly 100. We installed turbo. We have this file, so we've got some tasks and one of them is build, which is like literally go through all of the packages and run their build script. Now, it will do the dependency graph.

[00:05:37]
Right, so you don't have to define that, right? It'll go look for every one of your packages as defined in this pnpm workspace. So it's going to go look for package.json in all of these places. Right? It's going to look at their dependencies, it's going to look for the ones that have a build script, and it's going to run those in the correct order, right? For what it can parallelize, it will parallelize, for what needs to happen in a sequence because there's dependencies, it will set up the dependencies, and life will be good.

[00:06:09]
And, you know, like you can also have a type check which is going to do the same thing but type checking all of them, because like, for instance, depending on how complicated your system was, maybe you need to like build everything first so you can generate like the TypeScript types, you know, or something, right? I do a lot of like code generation tomfoolery, right, which is like I have, I use the TypeScript compiler and I will grab the types.

[00:06:39]
I will like set up my database tables using Drizzle, which will then create types and then I'll use a TypeScript compiler to take those types and create Zod schemas, and I'll use those Zod schemas to then like do contract testing and so you can get complicated. Right, but if your issue is builds are not in right order, this will also solve that out of the box the way it is. You can start out as easy as you want, and depending on, you know, how much time you like to light on fire, you can get crazy with it too.

[00:07:15]
But here we're just saying like, this depends on running the build script, when I do turbo run build, it depends on, you know, all of the packages being built first. For the outputs, which is where it's going to try to look for like, are these, should I, how, what am I hashing here, right? In that case, it's going to look in the dist directory for all of them. So you do have some alignment. If you have one package that's building to a build directory and another one that's built to a dist directory, you could add more options here, but I would argue stop that.

[00:07:46]
Right, fix that. That's like a five-second fix, just go fix it. Type check will go ahead and type check all of them. This is like world's simplest version and like having read all the docs top to bottom. There's not much more than this, right? One of the best things about teaching these courses is like, I kind of use a tool every day, but I'm like, I'm going to talk to people about it. I should read the entirety of the docs.

[00:08:10]
I can tell you there's not that much more here, and you know, linting will, you know, it'll run that across all of them, so on and so forth. But then the interesting one is, again, like, for unit tests, obviously run the unit tests, but like, let's say hypothetically that the testing we're doing in this workshop was Playwright tests and let's say you wanted to like build the preview app, you wanted like build all the assets, fire up the app, and then go do it.

[00:08:41]
So like testing will go ahead and rely on everything being built first, again, in that same dependency graph, linting we don't necessarily care, right? Like, and so on and so forth. So like type check we'll make sure we run all the individual type checks first and then we'll figure that all out. Testing, we do want to build everything first, and then dev is the interesting one which is like, don't do anything here.

[00:09:07]
You know what I mean? Like don't cache my dev server, please. You know what I mean? It's going to stay running, so don't like wait for the dev server to finish ever because it's going to like start up. Like the dev server is the one interesting one because it's not necessarily part of your build chain. It is, you're just saying basically like, do not wait on this, do not cache it. Effectively ignore it.

[00:09:44]
We're going to run npx turbo run build. Great, and you can kind of see all of my different, the dashboard's build script and the user's build script and the analytics build script, and the legacy app's build script, and so I went ahead and it built all of them, and that took 5.316 seconds now. Assuming that I haven't messed anything up, because again, I didn't even look at my notes. Oh, look at that.

[00:10:17]
It didn't rebuild anything. Right, because it built everything, it cached all of those things, and now this one took 2.93 milliseconds. Right, now imagine we had Playwright tests. Right, because like, you're like, great, five seconds, like, honestly, who cares? Playwright test, right? The idea that, you know, to your question from earlier, right, like, you can now only run the stuff that actually changed, right?

[00:10:44]
And like, that's incredibly powerful, and again, when we set up the remote caching, like, if Evan ran it and nothing has changed, like, you know, and then you run it, you won't even have to run it. You know what I mean? Because it's like nothing has changed since the last time, thereby, we don't have to run it. And so then CI/CD runs like that, if, like if you had a pre-commit hook and anyone ran it for the things that have, like if you're working on one slice, and where it's more important is that team topology, right?

[00:11:19]
Which is now your team is not slowed down by the existence of tests from another team, right? Because ideally, right, like, if you didn't touch their code and they've ever run the tests, last time they touched their code, you don't have to run their tests. If you did, if something, if you are the bottom layer that everyone depends on, and you change something, you will run everyone's tests. You don't get that with the individual repos, right?

[00:11:50]
Like if everyone's got their own repo, and you are the design system team, and you change the props on an accordion, how do you know you didn't break every other team? Hope. And that's why nobody ever updates their design system, right? Because like the team's like, we made this custom button that we weren't supposed to, and we didn't put it as part of the design system, and now we broke it, and the design system team's like, we don't actually know how our design system's being used, so we can't change anything.

[00:12:15]
And that's how you end up with 37 commits to make a button from green to blue or whatever it was. Don't live like that. Right, like, the problems that are hard are not just ways of life that you have to deal with and whine about. There's probably a deeply simpler way to do this.