Testing for Invalid Input

[00:00:00]
>> Steve Kinney: But we can start out with, it should,
>> Steve Kinney: Treat, do we wanna blow up? Yeah.
>> Female Student 1: If it concatenates things. But I mean, that's not how it's supposed to work logically.
>> Steve Kinney: You get to decide how it works. How do you want it to work?
>> Female Student 1: Key concatenate two strings.

[00:00:20]
>> Steve Kinney: Do we want it to?
>> Female Student 1: Why not? [LAUGH]
>> Steve Kinney: Okay, is that mathematical?
>> Female Student 1: No. [LAUGH]
>> Steve Kinney: Let's assume that it doesn't. Let's assume that we want ours to be better than the built-in function, right? So let's say that it should parse strings into numbers, right? Okay.

[00:00:44]
>> Steve Kinney: Again, these are things and if more of these ideas come up, let's just interject, and we'll do them, right? Again, there's a lot of times where you have a bug when you've got good test of code, is where you just didn't think about something, right? And so that's flow of bug comes in, you would write an additional test, you'd make it pass.

[00:01:01]
So it should parse strings into numbers. And again, this code doesn't exist. So I start out with a statement of how I would like the world to work, and then I will go ahead and make the world like that. So we'll say 1 and 1 and I think if I just did this in regular old JavaScript, I'd end up with 11, right?

[00:01:21]
But I want this to be two, cuz maybe these came from a query param where they're strings but they are spiritual numbers. Or it came from an input field where it was a string but I really wanted to be a number or somewhere, my API gives it to me as a string and whatever.

[00:01:35]
So we probably wanna have that add function in there. That's what happens when you talk and type at the same time. To be 2 and this is the nature of that kind of red-green factor, right? Which it will fail. And if I scroll up now, we get our first kind of useful error message, right?

[00:01:55]
Which is, you expected 2, you received 11. That's not what you expected. Red sadness everywhere, right? So then what we can do now is we could say something like, and you can be like, I would refactor this. Good, you have tests. You can refactor it. If typeof a, can you tell I've written TypeScript before?

[00:02:22]
Then a is, no, cool, or what I should do, if I was a better person. So I'd save see my test still fails, but I haven't gotten there yet. We're still 11. If typeof b = `string`.
>> Steve Kinney: b.
>> Steve Kinney: And my test passed again, yay. Where else can my addition function go wrong?

[00:03:00]
All right, cuz part of the practice here is to just be paranoid about all the ways your code could break. What are other bad values that we could get in here? We now know that if we take a string of as a number that could possibly be turned into a number, we're good.

[00:03:15]
>> Female Student 1: Well that can't be turned into a number.
>> Steve Kinney: All right, let's do it, let's find out. And what do we want to happen if we throw in the word potato as one of the numbers?
>> Female Student 1: Tell us I'm gonna give it a number.
>> Steve Kinney: Okay, so we wanna yell at the person.

[00:03:27]
Okay, cool. It should get real angry,
>> Female Student 1: [LAUGH]
>> Steve Kinney: If you give it a string that cannot be parsed into a number. These strengths are single quoted, so I will dance around using apostrophes as much as possible. All right, and this is actually like a segue way as they call it, so I think this is good, let's do it.

[00:04:00]
If I add 2 and potato, what do I want it to be? Well, I wanted to throw an error, but I haven't told you how to do that yet. So, let's just see it fail for a second, and then we'll talk about how to throw an error, cuz that's literally the next thing we're gonna talk about anyway.

[00:04:17]
We'll start out with that. And you can see that the test fails because we got not a number. And what we really wanted was either 2 or something else to happen, right? And so the request is that it blows up, because why did you pass it a potato, right?

[00:04:38]
So we need to know how to do that first. So what we wanna do is, we need to navigate the issue that I said test fail if an error is thrown, right? So we could implement the ability to throw an error, that's not gonna make my test pass, cuz an error would be thrown and it wouldn't be caught.

[00:04:57]
And even if I did my code perfectly, my tests still fail. And this will be like, see, testing is not for me. I'm just gonna check it out myself. No, we're not doing that. We're better than that. So we had this to be, and that was very easy for our very simple ones, but what if we said,
>> Steve Kinney: What if we expected it to throw, right?

[00:05:21]
There is one nuance that I need to draw your attention to, which is just because of the nature of the way JavaScript works. If this were to throw, it would actually blow up the test, right? Because it's inside this test function, what we need to do is just wrap it in an anonymous function.

[00:05:43]
>> Steve Kinney: And maybe keep the right number of parentheses. Right, so now effectively our test suite is gonna get this function here, which is gonna call that function. And it's expecting that, when it calls this function, right? That it will throw. And If we run the code, we can see that it fails, why?

[00:06:06]
Cuz it doesn't, right? Expected function to throw an error. And it doesn't throw an error, so we're red. That's not necessarily a bad red, because we haven't implemented that code yet. That is stealing ourselves for occasional test results because we're working towards something. So now, if I go back to this, here we have attempted to take strings and turn them into numbers, right?

[00:06:41]
And so there's a chance that even after that attempt to do that, one of them ends up as not a number, right? In the case of potato. So in that case, we can say.
>> Steve Kinney: If,
>> Steve Kinney: Second argument is not a number and our tests pass. But this is where we've got to train that thought process again, which is, where's the gap in my logic?

[00:07:31]
I say, if b is NaN, throw an error. What could technically still be garbage input?
>> Steve Kinney: A, [LAUGH] right? There is a practice in pair programming, which I refuse to do because I'm an introvert, right? Where the game is, one person writes a test, right? The other person then goes to write the most ridiculous, naive implementation that would get the test to pass, right?

[00:08:04]
And so, let's say we only had this first test, right? The implementer, the person playing the game with you, might just have a function that returns four no matter what. And your test would pass. And so then the game, the other player then writes another test. Well then, three plus two is five, right?

[00:08:28]
To make it fail again. And the goal is that you should end up with really well tested code, or a new enemy, a new arch nemesis at some point. So, this is thing, we solved one case. But then there's a part that's not really about writing code in any way, shape or form, right?

[00:08:46]
Which is to simply stare at the code and perseverate on all the ways that it can go poorly for you, right? So, yeah, like.
>> Steve Kinney: Give it.
>> Steve Kinney: a,
>> Steve Kinney: Second argument.
>> Steve Kinney: And so then we could theoretically go ahead. I'm gonna put this one beforehand because I feel like that's how order works, and where we just switch the arguments.

[00:09:19]
>> Steve Kinney: All right, our tests fail. We go ahead and we deal with that use case.
>> Steve Kinney: And the other thing we can do with that toThrow is, a test that is flaky is bad. A flaky test is one that fails randomly for reasons you don't totally understand. Those are bad cuz what are you gonna do when that happens?

[00:09:59]
You're gonna turn it off, right? You know what's worse than that?
>> Steve Kinney: A test that passes when it shouldn't, right? And so, sure, we're expecting this to throw, but this thing could blow up for any number of reasons. I could make a typo and this function would blow up, right?

[00:10:20]
And so what we might wanna do is make some expectations on, okay, what was the error message, right? So I could say,
>> Steve Kinney: What's it angry about?
>> Steve Kinney: When I decided to use a period and not use a period, right? And so you can see, okay, I expect this function to throw with the error that I expect it to throw with, right?

[00:10:53]
And now it's gonna be, I expect this code to blow up and I expect it to blow up with this error message. I forget if you have to use regex or not, let's find out together, right? You can even just do part of the string and it'll be I expected to have this that way.

[00:11:12]
If it's like, you put the time in there, or something like that, you don't really care about that. Or if there are other parts of that you don't necessarily care about cuz yes having it match exactly that error message is great until you decide you wanna change the error message, right?

[00:11:28]
Or until you wanna include extra information that could theoretically change so on and so forth. You wanna be like, I would like to make sure that at least this part is in there as well. And start to build up all of these places where things can go wrong.

[00:11:46]
But now the nice part is over time, again, simple code, very easy to see. But in a very, like a code base has got hundreds of files, right? The idea that you can constantly hammer at every edge case of any given function and know within seconds, right? Under a second, usually, unless it's a really big test suite, that everything works the way you think it does builds that confidence over time.

[00:12:07]
You're like, I changed something, 4,000 tests ran in five seconds. Nothing blew up. That doesn't mean you didn't make a mistake, but it just lowers the threshold of believing that you made a mistake, right? And so there are a whole bunch of other ways that these can go wrong, like I said, undefined.

[00:12:27]
Let's see if I can find it quickly enough. Even off the top of my head, tried to come up with a list of all of the things that I might think about testing for. Cuz again, this is Javascript You can throw anything in there. Could have been a string, could have been a number, and some of these are you deciding, right?

[00:12:47]
We know that true plus true equals two.
>> Steve Kinney: Are you okay with that? If so, great. You should write a test verifying that true plus true equals two, if you wanted to blow up because you expected those to be numbers, then you should write a test for for that.

[00:13:01]
There's not necessarily a right answer and what it should do. It's like, what are you trying to accomplish here?
>> Female Student 2: What's the the general rule, if you have a user input that you definitely have validation for in your code, do you repeat that validation in the test case?

[00:13:17]
Or do you assume, I mean, where do you draw the line?
>> Steve Kinney: I mean, I would test the validation logic, right?
>> Female Student 2: That's true.
>> Steve Kinney: Yeah, I would take that whatever's valid or invalid, cuz that is, even if that is deep inside of a angular component somewhere, something that's looking at that password field, right?

[00:13:34]
Maybe it's line 386 of a giant login form. What I would do. I work on open source code base, I'll show you some of this later. It's interesting because on my team I have one member who will shove everything in that component. And I take it to the other extreme where I have a utils folder that is probably to the other end of it.

[00:13:59]
Where I will take validate that there's an uppercase letter and make that a utility function that I test, validate that it has a number right, and then validate password is. Those four or five functions, all of them, but I've pulled out each one, because the smaller you break stuff up into in that logic.

[00:14:21]
When your test suite blows up, you know exactly which tests blew up, right? If all of a sudden, if it's a well-named tiny function that just checks to see if a string has a capital letter in it, and that test starts failing. Well, you know what went wrong, right?

[00:14:38]
Very easily and you get a lot more granular stuff in there.