Visual Testing with Chromatic

[00:00:00]
>> Now let's get to visual test. Now one thing I will say is this portion of the show is totally optional, but it does involve at least getting a Chromatic account. There's a bass line version of Chromatic which is totally free, up to like 5,000 pushes or something like that a month, whatever.

[00:00:25]
Maybe tests, I don't remember. Now it is hard for me to walk you through signing up for a Chromatic account when one has a Chromatic account, and setting up a project and stuff like that. So there is a Illustrated guide to that here. But let's kinda go over to chromatic.com, sure, /start.

[00:00:50]
That seems good. Choose from GitHub. We'll choose anthology. What kind of project is it? It's a storybook. So you can install Chromatic. I do think that I got that for free with Storybook, but I'm gonna follow the rules here for a minute. So it was already, no, we added it.

[00:01:15]
Cool, cool, cool, and then there's an idea of a project token, which I will delete later so I don't blow through my account here as well. What this will do is it just attributes this storybook with Chromatic, I think. So what Chromatic is used for is visual regression testing, but you do need to store what they looked like last time to see if they look different this time, right?

[00:01:47]
Do you think I got rid of that broken test? And it will run through all of your stories. And then the first time around it basically tries to figure out like, what does everything look like? And so now you can have a work flow where you can work through everything and see, is this different or not?

[00:02:04]
Sometimes different is good, sometimes that's why you opened up the code base. Ideally, sometimes when we write code, we seek to change something, right? It's just, did we change the things we wanna change? Or even I showed you that code of our button component, which is a mess.

[00:02:18]
Don't look at that code as an example, the code in this repo look good, look at it. Our button is a series of choices over time. And so one of my missions next week or so, is to sit down with the visual regression testing now the dark mode has settled down, and go through and set up the visual regression testing and then just refactoring.

[00:02:38]
I don't want them to look any different, but like, sure. Awesome, yeah, I can set up an environment variable and all that extra fun stuff, but we're not gonna do that just yet. And look, in Chromatic, here are my components. We got a button, we got the callout, and we got the text area.

[00:03:08]
Those are the ones I have stories for currently. And so, we can do something and ideally we should see a change. But the really cool thing in Storybook 8, I think this is somewhat new. What do we wanna mess with? Button or callout? Don't all answer at the same time.

[00:03:27]
Call out? All right, so we'll hit this. And what's super cool is this is in development. I could run it, I could do it on every push. We do it on our website, we have it in the CI CD. But what's super cool about this is, I'm in development right now, I am working, working.

[00:03:50]
And so, what I can do is let's go into the callout. And let's begin just something, right, like text. Let me put that in the actual class name, text-2xl. I could hit Play as I'm working on this. It does a little bit of updating and stuff along those lines.

[00:04:32]
It runs the test. But the nice part is you can actually see the changes and approve them. Cuz sometimes you do want, visual regression, I guess, that's why they call it a visual test. All right, we usually call it a regression, it implies that I didn't want it to change.

[00:04:47]
All right, in this case, you can kinda see, is there an easy way to make this bigger? Unclear, but there's always an easy way to make it bigger by clicking and zooming. But it will kinda show you an overlay of the old and the new component, right? And so here you can see that obviously, it's a little taller now that the font is in fact bigger, right?

[00:05:11]
And obviously, this was a very simple change. But you can see like, hey, this component has changed from whatever the last time you push stuff into main and approved things. And so you can actually say, yeah, that was intentional and accepted our whoops, didn't mean to do that, right?

[00:05:30]
And it's a great way for checking, obviously, the changes you want actually happen as consistently as you want it. Cuz let's say you were intentionally changing the button or some iconography, and you started to see some places where it didn't change that you thought it should. That's useful information.

[00:05:45]
Or did it actually like, I didn't even realize that we're using this icon at that end of the code base and I just removed. Something along those lines, be able to catch all those things on a visual level. Like yeah, if you get really fancy with the types, you'll get some of it for free.

[00:06:01]
But I think being able to catch it here as well, so I can hit Accept, and I can push that up next and this becomes the new accepted way to have call outs, right? And you can see that I can either batch accept, because, yes, all the callouts changed, right?

[00:06:20]
And it caught that, but they all changed because I changed the font size and the actual parameters. So I can also go ahead and either accept all the changes as one component, or everything, right? And so whatever you need is the right choice in this case, and then you can choose to rerun the tests.

[00:06:39]
And ideally, I think, since I accepted them, I think that pushes them up. Normally it's running in CICD so I don't think about it too much. Cuz we do have it set that you do need to go and approve it in CI. I think a lot of this functionality is relative, I think it went into beta not too long ago.

[00:06:56]
It's officially in Storybook 8 but we've been running in CICD for a while. So that's normally how we've done it. But the concept is the same whether or not you choose to run it through the GitHub action, going to the Chromatic UI, or in this new kinda nice way to see that.

[00:07:09]
Now there are no changes, cuz this is what we accepted a call out should look like. If I reverted this change, we'd get another visual regression change. Chromatic is not free, but I think you get something like 5000 comparisons and then something along those lines. You can choose to use it, you can choose not to use it.

[00:07:29]
If you can trick your employer into paying for it, that's even better. So I don't wanna belabor it too much, I kinda wanna point out that one of the workflow is really great, two, it exists, three, it's useful. But like I said, I don't come here to sell you anything.

[00:07:40]
I don't work for them.