Code Coverage

[00:00:00]
>> Test coverage, what it basically does is it statically puts little checkpoints throughout your code and triggers which ones of those got flipped to show. As we ran the test, what part of the code got exercised and what didn't, right? And it can be weaponized against your colleagues, and it shouldn't be, it can be more of a helpful tool.

[00:00:17]
A lot of times, could you break the build on it? You could. Should you? I don't know, it depends. There are places in my code base, we can talk a little bit more, maybe during the question and answer time, about how I choose to structure my code base.

[00:00:37]
The problem is I've been in charge for so long that my code base is structured the way I like it, and who knows if it's good anymore, thats why. So why you need junior engineers on your team cuz I don't have a feedback loop anymore. And so I, for instance, like to take all the functionality of my view layer components.

[00:00:55]
I have a giant utilities directory full of every little helper method that I could ever want all in one place, and I just rely on IntelliSense to figure it out. I don't have a deep-nested folder hierarchy or anything like that. And for those little like, hey, I got one thing that makes things into kebab case and another thing that pluralizes, lots of little stuff like that, like that folder?

[00:01:21]
Yeah, I would like to know that I've got 100% test coverage on, for other pieces, I don't necessarily worry about it. What I do use code coverage for a lot is, especially as I'm developing, as I'm thinking about something, what am I not thinking about? Show me from a different perspective, cuz a lot of times, when we're working on something, we've lost beginner's mind and we're just so in the weeds that we're not seeing it clearly.

[00:01:44]
So one of the things you can do is you can run it with this --coverage. And at the beginning, it will run through your test suite as it always does, and then you will get this. Now, to be clear, you're could have 100% code coverage and still have bugs, right, like your code.

[00:02:01]
[LAUGH] You could cover it all and just not have thought of that edge case, right? I'm not going to give you a book summary of the black swan, but that is basically the idea. Having 100% code coverage does not mean you don't have bugs, and having 0% code coverage does not mean that you do.

[00:02:15]
It is a useful tool for seeing whether or not you do, I think sometimes the more important part is, this view is great, however, you will see the red, you will get salty. I think what's more important is that out of the box, there are different reporters, you can have it report to JSON, you can have it report in this fancy table, you can have it whatever.

[00:02:35]
One of the things that you get for free, unless you configure it to do otherwise, is it will, in a coverage directory, which I have git ignored for reasons that should be relatively clear, you get an HTML output, right? Like I said, I will look at the red ones cuz I'm a bad person.

[00:03:03]
But generally speaking, where I will mostly go is if I had a utilities directory. If I had something where it's like this seems awful, I'd be like, that's an important part of the code base and that number's somewhat low, right? Sometimes in a bad mood and I will run it and then I'll hassle my co-workers.

[00:03:17]
But that's not my best self, so we're not gonna talk about that version of me. So obviously zero, there's nothing to look at there. What I'm sometimes currently interested is yellow, right? And in here, you can kinda see, okay, like this one, yeah, there's not a lot going on.

[00:03:34]
But red is stuff that as we ran through all this stuff, we never executed these lines of code. And like I said, effectively the way this works, it breaks it into an AS, it takes your code. And imagine putting little console logs in every branch of a conditional, keeping track of where the console logs were triggered and where they weren't, and using those stop and endpoints and then annotating the code with that.

[00:03:58]
Even if we go back into the revisited one, index looks interesting, you can see. Okay, when I refactored earlier, this thing, cuz I put it in a component test, right, and wrapped it in a provider, is 100% test coverage. It's loaded, we're doing all sorts of stuff with it that just the provider around the thing is not.

[00:04:19]
Do I feel the need to stop what I'm doing with my life and this thing that just wraps it in a provider but is otherwise chill, right? And I actually think I'm actually mounting it in the app. If this is a real application, I might even get it too.

[00:04:33]
But this is a good point, would I stop what I'm doing when I know that this isn't important if I'm testing this? And this is only going with vitest. So the question we got after the break, if I knew that the application, that this was like the first component I mounted, and I know that playwright is visiting it and just not in this report, would I stop what I'm doing for the sake of getting to 100?

[00:04:53]
I would not, right? I think that's the important part. But sometimes you will see that, there's one conditional in here, which is kind of important that I am not hitting. And I think that that is a lot of times really useful as well, right? Things that gives me a lot of happiness right now, it looks like the entirety of my reducer is being hit.

[00:05:14]
Reducers, as we saw before in that earlier example, function, stuff goes in, stuff comes out. If I'm just not at all dealing with a certain action that can get fired, I should deal with that, right? That's a thing that's gonna happen in my app. At no point do I say that I'm dealing with the removing of an item from the packing list, I should deal with it.

[00:05:33]
So it's less of a get to a 100, get to whatever number, and it is more of this can be a useful tool. Again, if you are trying to figure out like, I don't feel comfortable about something, right, then you can figure out where the hurt is. And so this one, for instance, this is super interesting, these are my hooks that I use in that application.

[00:05:56]
And there's an interesting thing that comes out here, which is looks like there's no test coverage for use all items. Interestingly, I didn't write tests for any of these hooks, it's a sample app. All right, I wrote some tests to prove whatever point that I wanted to prove, which was the rendering a component and using the component test to hit some buttons on it.

[00:06:22]
And you know why there's no test coverage for this? Cuz I don't use it, it's not actually used, right? And so even as we fire up the component, cuz you get a lot of test coverage for a component cuz it loads up everything and we hit a bunch of things, I didn't call this function ever.

[00:06:38]
This has no test coverage so I didn't use it, which means, you know how I get this file to 100% test coverage?
>> Delete.
>> [LAUGH] I would delete the function rather than write tests for something I don't use, yeah
>> Probably not important, but I'm wondering about line 15.

[00:06:54]
How is it that an end bracket wasn't being hit?
>> I don't know.
>> [LAUGH]
>> The idea that you know at one point, there's I think, in the directory, you can see the data structures and makes like the fact that this stuff works at all. But this, I don't know why line 15 wasn't hit.

[00:07:15]
At the same time, right, it also shows you why trying to get to 100% is silly.
>> Sure.
>> [LAUGH] Right?
>> So my first thought is key, it's not that important,
>> Not important, but it is useful. It is less about any given number on a given row, and more about, weirdly for a relatively technical workshop on maintaining large apps, there's a lot of appeals to your gut feel, right?

[00:07:44]
Can you update a dependency? Do you feel comfortable shipping stuff? If you look at this chart, and like this one again is slightly ridiculous because it's an example app. But the fact that when you look through this, and again, fake app, but when you look through this, does anything look off to you, right?

[00:08:03]
There could be entire parts where it's like, you might have a folder of JSON data. Yeah, that's probably not test covered cuz yo, and you could sit there and tweak it, but I don't know if it's always worth it. In this case, the solution has 100% test coverage and the exercise doesn't, right, and stuff like that.

[00:08:21]
But generally speaking, I think this is a really great view for thinking about things, I wouldn't hold it as heuristic. However, like I said, there's some times where it's like, yo, my utilities directory. If one of those things throws an error and takes down the app, that's problematic, right?

[00:08:37]
Things that are easy, you might choose to enforce a threshold. I will be totally honest with you, at this moment, we do not currently enforce a threshold on my team, right?
>> Like that 80%
>> Yeah.
>> Specifically?
>> Yeah, right. And I have the weird situation where I work at a tech company that makes tech for developers, and the CEO and the CTO are the creators of the technology, and my CEO doesn't believe in 100% test coverage.

[00:09:09]
He believes we should have tests, right? And so that is nuance to us, our kind of view on this. And mine doesn't look this red, to be clear, mostly cuz one of my fun thing to do during meetings that I don't care about is fix Lint rules and try to bump the test coverage up on a file, almost like a Sudoku puzzle.

[00:09:26]
So mine is shockingly high for someone who says he doesn't necessarily follow it religiously. I'm a little bored right now, [LAUGH] maybe I can get this up a number. And that's a fine reason to bump the number up as a game, or where you feel it's important, but it's useful.

[00:09:43]
But let's talk about if you did want to enforce a certain rule, I do have some heuristics, cuz again, there are time and a place. There are certain files that I feel are critically important that I check, and if they do drop below a certain amount, I am gonna have some to-do items.