Performance Budgets as Constraints

[00:00:00]
>> Steve Kinney: You know, another thing is, there's that like Peter Drucker quote, what is measured is what matters, right, which is, you can have whoever just, you know, like, obviously performance is important, right? But unless you have checks, it is going to drift until it becomes problematic, right? If you do not have checks for like, are we running an accessibility test, right? You will find that, you know, even if you have people who are passionate about accessibility, like, you know, my team, or my former team is, I was like, my team was, they're still passionate about it, I'm just not on it.

[00:00:41]
But like, you will find yourself in positions where like, over time it drifted and your accessibility is not great anymore, right? And so it becomes like, how do you build these things in there? And I think like that has to be something that happens probably us again at the CI/CD level. Now, where the nuances become interesting is. You'd be like, well, we care about performance, and I want to check performance as part of the CI/CD process to make sure the bundle hasn't grown, and the root level chunk is the right size, and we get at least, you know, largest contentful paint in 2.5 seconds, and our CI/CD our Lighthouse score is up there and all of those things, right?

[00:01:23]
You, all those things are going to be very important to you. The big question that I challenge you with is. How often do you need to run that? Right, if it's like, do you need to run it every time you push to the PR branch. Maybe. Right? Or is it like, you know, I think for most job runners, like GitHub Actions does this, I know BuildKite does it, you can also have jobs that run on a like cron schedule, right?

[00:01:54]
Of like, maybe it is once a week we run the performance test, right? Once a sprint, like the day before sprint planning, we run the performance test or the accessibility test and stuff along those lines. And yeah, you need to then like make sure you look at it. But figuring out like, does everything need to like block a PR all the time, right? Or is it a monthly check-in to see has test coverage dropped?

[00:02:19]
Has our performance like changed, and all of those things. I think are like also, also like somewhat like important things to consider about the cadence in which you do it, because if CI takes 45 minutes, nothing's getting done, right? It's about that feedback loop, because if it's like, oh. I forgot to adjust the CSS or tweak this thing and you had to wait 45 minutes to find that out. What happens to your velocity at that point?

[00:02:47]
And then what do you do? You start turning things off. And what happens after you start turning things off? Things degrade over time, you know what I mean? And so it's about figuring out how do you like manage and navigate these tools, so that you are keeping yourself accountable, and like, what is the system, like if you're using something like Jira or Linear, is it, hey, it runs once a week and we built the API integration here, so that like, if it drops below a certain level, an urgent ticket is made and put into the next sprint.

[00:03:18]
Right. The specifics of that obviously, unfortunately are going to be like for your team, that's like what I do for myself, to be clear. And like, I have actually like Codex on a nightly cadence, like, run a job, analyze the result, and I wake up in the morning to an urgent ticket. Right, if it's dropped below something, right? That works for me. What works in your like system, like, it might be like, yo, we're not letting in like OpenAI read our entire codebase once a night, thank you very much.

[00:03:47]
You got to figure out something else, right? But I think what that version looks like will always kind of depend as well.