Types of Tests

[00:00:00]
>> As you can kind of guess from the preamble, the majority of our time together is kind of on that systems piece and like, yes, could you write build processes for all sorts of things? Sure, but the lowest common denominator of the thing that you should be automating is likely and enforcing, is likely does your application work, right?

[00:00:21]
Once you have hit that level of the hierarchy of needs, you can move up to other ones like, is the title of their pull request formatted in a way that automatically closes the JIRA ticket? When you're solving those kinds of problems, I love it, but you gotta you gotta solve the baseline problems first, right?

[00:00:40]
So the problem with talking about testing is that we need to at least spend a few minutes together coming up with what is going to be in our time together our language of testing, right? Because there are a lot of terms those terms mean things and we use them wrong all the time.

[00:00:57]
And most of these terms have been used wrong so much that sometimes the wrong way is more important than the right way. So most of the first section is, let's get on the same page when I say words, what they mean. And then we'll get kinda comfortable with the very basics of some of the tools that we're gonna use.

[00:01:13]
Disclaimer, these are my definitions, and since I'm gonna do the majority of the talking. [LAUGH] Just so you, we have a common frame of reference to the things that I'm saying. These do not represent the opinions of my employer or anyone else in that matter, particularly anyone on Twitter, right, don't at me.

[00:01:34]
There is a wide range of tests, they have different sets of trade-offs. Anyone that, it's not fair. I would be hesitant to say it should these are the best kinds of tests, there are trade-offs, right? At my very scientific diagram here, you can see that you've got on one end unit tests.

[00:01:55]
You know what's cool about unit tests? They run really fast, right? Sometimes like if you have the right editor plugin, they'll like run in your editor and put a red squiggly line or something like to the left on the sidebar almost immediately, right? A lot of these are like put two things in function, see if the thing you expected to come out came out, right?

[00:02:15]
That's really easy to run, like V8 super good at that kind of stuff. That said, they are testing to see if you put two things into the function, so the thing you expected to come out. Yeah, that that's great to know that but you know who's not doing that?

[00:02:31]
Your customers, they are clicking buttons, right? Usually the most hostile way that they possibly can, they are clicking buttons from the web browser. So all the way on the other end, we've got to spin up a browser and pretend to be the user clicking buttons. Arguably, if you've got really good tests in the sense, right, that are doing all the things and exercising your application, you can be super confident that you didn't break anything.

[00:02:56]
That said, as you are writing code, you're probably not running these all the time, right? Because spinning out the browser, navigating across, hitting the buttons, it's not slow, but it's not fast either, right? And sometimes you think about, what is the feedback loop that you have? And if you can get immediate feedback that the thing you did was bad, then you can make the next decision, right?

[00:03:16]
If you have to wait four minutes, every time you make a decision, the number of thoughts you could have in a day is way lower, right? And so there might be people say, if it's covered by this test, you shouldn't have unit tests, I don't believe these people.

[00:03:30]
Sometimes I wanna know real fast, sometimes I wanna know real confidently and I'm willing to wait a few minutes, right? These are both fair things and sometimes it may be I have two tests of different kinds covering relatively the same functionality because I'm two different people. I am both the person on call sometimes, and sometimes I'm the person trying to write some code, right?

[00:03:54]
And those two people have different needs, and so sometimes I will break the rules and have different kinds of tests testing the same thing. And I think that that's totally okay. And we'll talk about the one in the middle when I get to the slide for that. Like I said unit test smallest possible testing get away with and we get back what we expected, that's a joke.

[00:04:16]
That's the only pun that I'll be making today intentionally there was mark the time. And there we go. Then on the other end of spectrum there are end-to-end tests. And I mean real end-to-end tests, like true end-to-end tests, right? Testing everything from like the backend to auth the whole soup to nuts, kicking the tires on your application.

[00:04:43]
These are great, these are wonderful, I have never seen these in the wild, right? Because there's a lot of infrastructure, and particularly on the front-end team, you probably don't control a lot of it, right? So for instance, to have a true end-to-end test, you need the ability to spin up a brand new user with a certain settings whether or not they are pro user and have a credit card and their whatever.

[00:05:09]
And then tear down that user, right, and get everything back exactly the way that it was, right? There's lots of reasons why you don't have this. When I worked at certain company that sent emails, what we had was a pool of test users. So only one test runner can be running at a time because if two tried to use the same user they would both fail and probably leave that user in an unknown state and then your tests would just break for perpetuity, right, not great.

[00:05:41]
A lot of times you can't really test truly against production because especially when I worked for a public company things that like made the data not real, were bad, right? So there's all sorts of reasons why true spin up a brand new database end-to-end doesn't exist. A lot of times even on a large enough company spinning up everything on the fly is not a thing that you can do, right?

[00:06:09]
And if you can, we aspire to do it at Temporal, but we can get away with stuff open source we own the server so on and so forth, right? We still don't have it, but we are aspiring to do it. But generally speaking, this is the ideal.

[00:06:21]
But also you can imagine these are expensive and they are slow. So they come with some stuff in the middle. Then we have the word integration tests. This is the most hand wavy definition that we're gonna talk about today. Integration tests is like one or more units, right?

[00:06:40]
Which is like almost everything, right, after the various put two things in a function, look at the return result, right? We're going use this to refer to our spinning up Paywright or Cypress. Something that's kicking the tires on our app in the browser, even if we have to mock out some of the APIs or something along those lines and kind of have it work in there.

[00:07:08]
We're gonna call those integration tests because they are testing to see if all of our components, when we slapped them all together in that UI, do they all act together nicely the way that they should? Is that truly an integration test, I guess? Are some of our unit test maybe technically integration tests?

[00:07:24]
Sure, I don't care. That's gonna be the term we use, cuz I believe in my experience that is generally are at least as Frontend engineers are agreed upon terms. I'm sure that someone will grab a time machine from the early 80s and tell me that I'm wrong, I don't care.

[00:07:44]
Lastly, component test, which is a totally made up term, which is those tests somewhere in the middle. Don't visit an entire page, but maybe you grab me that accordion and we open and close it a few times. Maybe we fill in some fields on that form of just that form component and see if the button is enabled or disabled, right?

[00:08:03]
These are somewhere in the middle of our very small unit tests for utilities or just JavaScript functions and our browser-based ones. These live kind of in the middle. They are faster than spinning up Playwright, but don't give you the same amount of confidence as literally just visiting the page, stuff along those lines.

[00:08:23]
As we know, CSS is a thing, right? And the box model is a thing. And how a component operates when you render it in isolation is different than when you put it in a flex box or something along those lines, right? And so you these are a healthy trade-off.

[00:08:43]
And on any given application, you can dial in how you feel about this. I've seen applications where it's a lot of component tests. I've seen applications where prior to testing library, we installed enzyme, we installed it. There is probably a component test in there somewhere, right? You can kind of choose your own adventure with this, it's about, what works for you and your team.

[00:09:08]
My job is to kind of at least show you how to set these things up, kind of take you through the paces of, what's effective where, what helps deal with what you wanna do, so on and so forth. And sometimes how can we abuse some of these tools to get test coverage when we don't have any, right?

[00:09:28]
Maybe that's a Playwright test that exercises your entire application. Cool, at least he got that. Maybe it's like, hey, you know that function that nobody really knows what it does and has no test we just know that we don't want it to change? Maybe that's a place for snapshot tests despite what you read on the internet about how snapshot tests are bad, right?

[00:09:47]
I don't know how it gets these outputs, but just throw a bunch of stuff in it and verify that it didn't change, right? Like I copy and pasted a bunch of API responses into a JSON file, shove them all in and make sure what comes out the other end is the same.

[00:10:01]
That's in a messy code base, somtimes you got to do what you gotta do.