Testing React Applications, v2 Snapshot Q&A

[00:00:00]
>> Kent C. Dodds: So I'm gonna talk a little bit about how you can take snapshots too far and what their real role is. But before I do that, does anybody have any questions or observations, yeah?
>> off screen male: So, part of render automatically generates a snapshot?
>> Kent C. Dodds: Good question, I'm glad you asked.

[00:00:21] So render doesn't actually care anything about snapshots. It's all very just, like, DOM focused and rendering React components to DOM nodes. So the snapshot, the job of the snapshot is jest and this assertion to match snapshots. So we pass a DOM node to match snapshot, and it will be responsible for serializing it and saving it into a snapshot file.

[00:00:45]
>> off screen male: So then, the first time it tries to do a comparison, if there is nothing, if there is no existing snapshot it will generate the initial.
>> Kent C. Dodds: Yep, that's exactly right. So, I'm glad you asked that also, cuz it would be good to see what an update would look like.

[00:01:01] If I change this to name, for example, then we'll see the snapshot fails. And it shows us what that failure looks like. And then it'll tell us right here, inspect your code changes or press U to update them. I could press U and it'll update my snapshot. Except now my other test is broken cuz it depends on this being username.

[00:01:24] So I'll change it back and my snapshot will now fail because it updated. So I'm gonna hit U to update it again and we're all set. And because we're including CSS, because of that snapshot serializer, if I go to form, where we're using glamourous for this stuff. And I align items left.

[00:01:44] Then I'm going to see that also will change. That includes our vendor prefixes, which happens at runtime. You see that's not actually written out, but it is important here, so. And we could update that with you and so on and so forth. So snapshots are not the only way to accomplish what we're doing, but it pretty much enables it.

[00:02:06] Cuz like I would certainly not want to copy paste all of that into my test file and then the serializer aspect of it as well. Where it formats it nicely for you and everything. So I'm glad we have snapshots, it's pretty cool. Yes.
>> off screen male: Do they contribute to the coverage report?

[00:02:23]
>> Kent C. Dodds: Yes. So I'm rendering the component. I'm running all this code to get that container in the first place. And so the fact that I'm running all that code will count it as coverage.
>> off screen male: So if you rendered an entire component, would that be [INAUDIBLE] if new component was just the HTML.

[00:02:43]
>> Kent C. Dodds: Mm-hm.
>> off screen male: Would that give you 100% coverage right off the bat?
>> Kent C. Dodds: Yep, it certainly would. And if your component's just like pure HTML, then that would be a potentially reasonable thing to snapshot. But if I'm looking at a pull request of a component that's just HTML stuff and then the snapshot, I'm gonna see two diffs.

[00:03:06] I'm gonna see the diff in the component that's just HTML. And then I'm gonna see an identical diff in the snapshot. And then I'm gonna wonder to myself, what value is that test really providing me? Because it would provide me value if it, like, I didn't realize that this update was gonna change this.

[00:03:25] But, no, yes, clearly I did because the diff looks identical. You won't even know, is this the snapshot I'm looking at or the component? So I wouldn't really snapshot something like that. Good question, yeah?
>> off screen male: Is there a way to undo an update of a snapshot?
>> Kent C. Dodds: Yeah, git reset.

[00:03:42] [LAUGH] Or git checkout that snapshot file. Yeah, jest won't keep track of those for you, though. So snapshots kind of in that vein are not really useful for TDD, test driven developement. Because unless you handcraft the snapshot, which when I'm doing the babel plugin, sometimes I do actually handcraft the snapshot, cuz I know exactly what I want it to look like.

[00:04:06] But generally you don't handcraft the snapshot. So yeah, it's not really normally a good candidate for TDD.
>> Kent C. Dodds: Any other questions? You're wondering what I have to say about effective use of snapshots? Let me show you an ineffective use of snapshots. And this is a pretty small app that we're running in here.

[00:04:31] But I have a snapshot of the entire app, the homepage of the app. It's a pretty small app, but this snapshot's 260 lines of code. When I first started doing snapshots on my team, we started getting snapshot files that were easily 1,500 lines long or way longer. And that was pretty cool.

[00:04:55] It's like, man, we will never not know that things are going to change. That's great. But you already know that because you see a git diff. Yes, things changed, clearly. The nice thing about snapshots is that you know what impact a change, like you change the button, all the snapshots are gonna break.

[00:05:13] And you can update those, and you're like, okay, that button's used over here, so it broke. So that's kind of useful, it's kinda nice. But what ends up happening, and Peter and I were talking about this, it's 3 o'clock in the afternoon. You're like, you just wanna make this pull request so people will review it, and so you're like, okay, great.

[00:05:32] It says seven snapshots failed, you're scrolling through the, looks good. You hit the U key, you get them updated, you push it up to GitLab or GitHub or whatever, people, now it's 4 o'clock and your coworkers are like, okay, let's just get this thing merged. Peter really wants to get this in.

[00:05:49] So I'm just gonna scroll through, yeah, it looks. And he probably was careful about this, so I'm gonna just trust him and I'll just merge this. And now you have a broken app because people didn't review it carefully enough. So that's kind of a team discipline problem. But if our software allows team discipline to break our software, then maybe we should.

[00:06:08] Or if our tools allow team discipline to break our software, maybe we should be using different tools. And so I have this blog post called effective snapshot testing. Here, I'll post this in the chat, I keep forgetting to do that. Hey look, George already did that. [LAUGH] Thanks, George.

[00:06:30] But I quote Justin Searls, who is a really big tester, especially in the Ruby ecosystem but also in the JavaScript ecosystem. And he explains several of the things that he doesn't like about snapshots. And basically says snapshots are worthless. I take a step back from that a little bit and I say, yeah, snapshots do have their places and there are a couple places where they really shine.

[00:06:57] So Babel plugins and with being able to snapshot CSS. So if you have various components that are style-specific, it can be nice to know when those styles are going to change and what impact those changes will have. And then it gives you a couple ideas of things that you can to improve your effective use of snapshots.

[00:07:18] So in general, what we're trying to accomplish when we're testing, well, there's the side of testing where it's like, okay, I'm testing to enhance my workflow, that's TDD stuff. But normally what I care about with testing is to bring confidence to myself that my applications are working in the way that I intend them to.

[00:07:39] And snapshots don't quite do that. Because when I say my application's working the way that I intend it to, well, this snapshot could break if somebody refactors and removes that. But my application's still working the way I intend it, right? So there's nothing that actually changes with that.

[00:08:01] That's actually kind of an implementation detail. And so there's no way for me, in a snapshot, to communicate what part of this snapshot really matters for this particular test. What's important here. And so because that's hard to communicate, somebody coming down the line to maintain your tests that you left around, they are unable to know, like, I broke a snapshot, is that okay?

[00:08:29] They don't really know if it's okay to break a certain snapshot. And so generally what I would suggest is try to query around the DOM to the specific thing that's important for your test and snapshot that. And what often happens is like, this test only cares that the image gets or this image gets the right source.

[00:08:54] And so I'm gonna snapshot that. And now I have an image. And it's got the right source in my snapshot. And then I'll take another step back and think, I could actually just change the assertion. Instead of two match snapshot, like get attribute source equals this source string.

[00:09:10] And just put that right in my test. So now I don't need a snapshot anyway. So snapshots are great. Just use them like with critical thinking about whether you really actually need to use the snapshot. Yes?
>> off screen male: Okay, so by explaining how we shouldn't use them, you're describing a lot of situations where there's a really high signal to noise ratio so that people basically get into the habit of ignoring the noise.

[00:09:45] And so that's the danger, I understand that. I'm still trying to wrap my, yeah, okay, so narrowing the focus of your snapshot so that you're isolating just the thing that you care about. Well, isn't that the point of any of the other testing tools that are built into jest?

[00:10:03]
>> Kent C. Dodds: Yeah, so you're suggesting, okay, so what's the point of snapshots if we're going to narrow it down anyway?
>> off screen male: And also, following along with the point that you have been going along with, the whole way has been, look, pay attention to what the user would see and what the user cares about.

[00:10:23] The user doesn't care about how it looks when the DOM-
>> Kent C. Dodds: What the DOM looks like.
>> off screen male: Yeah, what the DOM pukes out into text? They care about how it looks and/or whether there's a spinner or not.
>> Kent C. Dodds: Mm-hm.
>> off screen male: Whatever the label on an input or a submit button, so.

[00:10:43]
>> Kent C. Dodds: Yeah, no, I totally agree with you. And honestly, I don't use snapshots a whole lot. Recently, when I was preparing this workshop, I was thinking critically about this, and like, okay, how can I tell these people that snapshots, or how can I communicate that snapshots do have a place?

[00:11:00] And I actually tweeted, I'm like, I actually don't think that snapshots do have a place. And some people kind of convinced me that snapshots are useful if you can as a team decide that they don't replace good unit tests. And so you'll notice that we actually have like full coverage with this test.

[00:11:18] But then in addition, we have a snapshot. And so it's okay if your team is conditioned to just update. But I wouldn't condition the team to just update and not think about it at all. But having a snapshot in place means that if I make a change to one of these dependencies and I say, okay, now the font size is gonna be 17 and then my snapshot breaks.

[00:11:42] It might break a snapshot that I wasn't expecting it to break. I didn't realize that component X uses FieldContainer. Now I have to kind of think about that a little bit. So as long as you as a team decide, okay, these are not going to replace solid unit tests, but they're just here to supplement the existing unit tests to kind of help us catch those little cases like that.

[00:12:07] Then it's okay to not really rigorously review those snapshots, because you have a whole suite of unit tests that would catch the things that actually really do matter. And so that's kind of where I've landed on how I feel about snapshot testing with a UI is that its more of a just kind of a gut check that this is changing things and the things that it's changing I'm okay with.

[00:12:30] Yes?
>> off screen male: So is it like a snapshot can be used only for the UI elements? Or can we use the snapshot for some kind of utility functions or some transformation functions?
>> Kent C. Dodds: Yeah.
>> off screen male: Can we still use snapshot, or do you think it is just a UI document renders, we can use snapshot?

[00:12:48]
>> Kent C. Dodds: No, so in my example here right down here. So we're snapshotting an object. Here we're snapshotting, and that object includes errors and dates and all kinds of stuff. So yeah. So it can be useful for snapshotting lots of stuff. And actually, there is another assertion that I find quite useful.

[00:13:13] It's called, let's see, toThrowErrorMatchingSnapshot. So if you have an error message, you're like, it throws an error. I could say like, you wanna make sure it's throwing the right error. And so you're gonna have like a regex that matches your error or something like that. Instead, you just snapshot that.

[00:13:32] And somebody changes the error message, really easy to update. Like it's totally fine. It's a way to verify that. So I actually use this one a fair amount.