Configuring NYC

[00:00:00]
>> [MUSIC]

[00:00:04]
>> Kent C Dodds: And we're going to use a tool called NYC. So this one, there's not really an exercise associated with it, it's just configuring a tool. So you can follow along with this one or just observe. So here we're going to add a dependency right after mocha, called nyc.

[00:00:23]
So normally, you'd just install this using NPM, but it's already installed for you. So we have the latest version, 7.1.0. And to use it, it's actually quite simple. We'll just add it right here in our test command, nyc.
>> Kent C Dodds: And that's pretty much it. We do want to configure it, though, a little bit because actually, I don't even know what's gonna happen if I tried to run it without configuring it.

[00:00:53]
Yeah, it actually just works magically, which is pretty cool. But we want to also make sure that we're maintaining a threshold of code coverage in our project. Actually, sorry, I should stop. Has anybody gotten to this point, where you have the report and everything, you ran the tests and it's reporting?

[00:01:13]
Anybody working with me on this?
>> Kent C Dodds: Anybody on the chat?
>> Kent C Dodds: Sorry, just waiting for people. Okay, so yeah, once you get to this point, the report should look a little prettier, just like this. Let's say that we added some functionality here. Somebody adds a function. Yes, we've got some people online who are following along.

[00:01:45]
function foo, no, not autocomplete, foobar, and that returns foobar. And we'll just add that here, foobar.
>> Kent C Dodds: Okay, so then if we run, npm t again, we're going to see that we have 0% of our functions covered, which results in, yeah, I gotta fix. The output is all messed up.

[00:02:12]
Yeah, which results in a single uncovered line and lower overall code coverage than we want. So what we're going to do to enforce that we have a certain level of code coverage is in our package.json file, we're going to configure NYC. This is unfortunately the only place to configure NYC.

[00:02:31]
And it'd be really nice if you could have an NYC RC file somewhere, but you can't. So just stick it right in your package.json. And actually, it was a smaller library. I don't mind configuring stuff in package.json too much, it's not bad. So if you're following along, you'll create an nyc property on the package.json.

[00:02:55]
Here's a quick pro tip for you. NPM actually provides a mechanism for library authors to do something like this to allow you to configure stuff in your package.json, and it's with the config property. And the reason that they do that is so you can put nyc in here or whatever, like yourlib in here.

[00:03:19]
And then if NPM decides one day that they're gonna add a property called yourlib, you're not gonna conflict with them, because they promise that config, they'll never do anything special with config. But unfortunately, some libraries just say, well, the heck with it and just put it right on the root.

[00:03:35]
So you have to look at how the library does it. So yeah, for nyc, there are several properties that we're going to set in here. The first one that we're going to set is check-coverage to true. So when it's all said and done, running our tests and checking our coverage, it's going to verify that the coverage matches a certain threshold.

[00:04:03]
And so then we can set what that threshold is. So again, it is branches, let's see, yeah, we'll just say, branches we want to be 100%, functions, we want to be 100%, lines is 100%. Now hold on, I said in here it says function. I'm pretty sure it's functions.

[00:04:27]
Let me go verify that here in a little bit. But yeah, it'll be lines and statements, 100.
>> Kent C Dodds: And let me verify this thing really quick.
>> Kent C Dodds: Branches, yeah, it is functions, right there, boom. I misspelled it in the workshop repo. So note to self, I'll fix that so that you all can have it correct there.

[00:04:57]
Okay, cool, so with that, you should be able to run the test command again, and it will blow up because I like trailing comments, sorry. Your package.json must be valid JSON. So here we get an error. This script failed because our coverage for lines, function and statements did not meet the global threshold of 100%.

[00:05:22]
So that's cool. That way, as you're developing your library, you're adding features to it, and you're like, okay, sweet, I'm ready to go. You push this out, then CI fails. Or even better, you have a Git hook that runs all of your tests and stuff, and we'll set up that Git hook a little bit later.

[00:05:39]
But sorry, I've just realized I should probably check on the time here. Yeah, cool, we're doing actually great, right on time. So let's go ahead and we'll remove that extra function. We don't need that anymore. And we'll rerun the test. And boom, we have 100% code coverage, it passed.

[00:06:04]
And yeah, so we can release our library with confidence that every line of code can possibly be run.
>> Kent C Dodds: Cool, let me just make sure a couple of other things. So there are a couple of ways that you can have this report. So actually, let me put that back.

[00:06:29]
Let's say you run this script and you're like, no, I don't have code coverage. And let's pretend that you don't see this uncovered lines thing. I'm contriving this example to show you something. [LAUGH] So you can actually have NYC generate a report for you to look at what lines are being covered, yeah, to make it easier for you to add coverage to new lines and things.

[00:06:57]
And that is using the reporter field. So this, I believe, defaults to text, but when you set it, you'll wanna add text back. And that's what we're seeing right here. This is the text report. But it's not altogether helpful. It's not showing us all the information that we really want.

[00:07:18]
We wanna see in my lines of code, in my actual code, what lines are covered and what aren't. So we're gonna add lcov right here. And an lcov report, it's a standard for code coverage. I'm not exactly sure what that standard is exactly, but I do know that it generates an HTML report for you.

[00:07:45]
And so if you run the test command again, you should continue to get your text output because we have that text reporter. But you should also notice a coverage folder pop up in your directory. Inside of that, there will be an lcov-report folder. And inside of that, there will be an index.html file.

[00:08:08]
If you pop that open, I always just use the command line for this. So open coverage/lcov-report/index.html. So open that up in a browser and it should look like this. And this will have all the files in your project that have coverage recorded. So we'll say index.js, and we'll see, very clearly, this function is not run and this line inside of that function is not run.

[00:08:36]
But it looks like these lines were run one time, so that's great. And so it's very clear to you what lines you still have yet to cover. I use this all the time, both in work and in libraries. It's really helpful. Unfortunately, it doesn't work with watch mode, because the way that NYC works is it waits until the process exits, and then it reports coverage.

[00:08:59]
And so if you did watch mode, then it would continue to count those lines of code until you exit. And so potentially you can have, this was run 25 times. So yeah, that's unfortunate.
>> Student: Is this a report that you publish every time you do a release of it?

[00:09:15]
>> Kent C Dodds: That's a great question. So later on, we're gonna use a service called Codecov in combination with Travis CI, where we'll take the lcov.info report, which is just like this. Well, normally, it's an enormous file, but we have a small project. But it's a bunch of gibberish stuff that I don't understand.

[00:09:35]
And it will send that to Codecov, and Codecov will give a report that looks very much like this. And then you can get a cool badge on your GitHub repo that says, 100% code coverage, and you can feel pretty awesome about that. And even cooler with Codecov is that it integrates with your pull requests as well.

[00:09:54]
So when Travis runs the tests and stuff on your pull request and reports those to Codecov, then Codecov has a bot that will say, coverage will change by this percentage by merging this pull request. Super cool. Maybe we'll get a chance to see that, question?
>> Student2: They're asking if Istanbul is another code coverage testing tool, can we use it here in the same way that we use NYC?

[00:10:14]
>> Kent C Dodds: Yes, Istanbul is very similar. And in fact, a fun story, Istanbul was developed quite a bit longer ago or, yeah, a while ago before NYC was developed. And recently, with the release of 7.0 of NYC, we're now on 7.1.0, this was three weeks ago, Istanbul and NYC merged.

[00:10:37]
And now NYC is part of the Istanbul organization. And so Istanbul is actually being used under the coverage by NYC to instrument your code. And NYC is the official command line interface for Istanbul. In a little bit, when we add ES6 to our code base, Istanbul by default can't cover ES6 code.

[00:11:05]
It doesn't understand default args and stuff like that. And so we have to use a babel plugin from Istanbul. It's a babel plugin Istanbul, aptly named. And we use that to transpile our code or to instrument our code before NYC runs it. So yep, great, any other questions?

[00:11:36]
>> Kent C Dodds: Okay, sweet, so let's move right along. There is actually one more thing that is done for us in this next branch. We also add the coverage and NYC output folders to our gitignore. We don't wanna commit those. And we also add this include source to our NYC configuration, just to make sure that in the future, if we start adding other files and stuff, it doesn't start instrumenting files that aren't in our source directory.

[00:12:18]
We only care about stuff that's in our source directory. So it's just mostly protecting us for the future. We don't really need it right now, technically.
>> Student: And you're operating under a convention that test.js is identified as a test?
>> Kent C Dodds: Yeah, so-
>> Student: So it will not be instrumented and it's not gonna-

[00:12:33]
>> Kent C Dodds: Yeah, that's an excellent question. So because technically, index.text is in our source directory. And so NYC assumes a couple of things about your setup. You can configure it with these defaults. But if there's a test directory, then it won't instrument anything in that. You can also do something really popular in the React community, it's __test.

[00:12:58]
Underscore underscore, anything in that will be instrumented, anything with *.test.js or spec.js. And like I said a little bit earlier, I have a lot of preferences, and I like my preferences. And so if a library doesn't support my preferences, then I make a pull request so that it does.

[00:13:20]
And so, yeah, I'm responsible for NYC supporting this style of filename convention. So you can use .test.js, yeah. Okay, cool, and I would encourage you all to do the same thing. If you have preferences and you can argue your case, make a pull request to make it a reality.

[00:13:38]
I think that's a totally legit thing to do.