Exploring ASTs

[00:00:00]
>> Kent C Dodds: Yeah, so how do we actually use these things? So we're gonna be talking about three use cases for ASTs, Babel Plugins, ESLint Plugins and Codemods. And we're actually gonna start with the ESLint Plugin first and then we'll get in to the Babel Plugin stuff. So
>> Kent C Dodds: Here we have something set up and you can feel free to follow along if you like but I do have exercises that I'll be giving you to actually work with the AST yourself.

[00:00:36]
So yeah I,
>> Kent C Dodds: Feel free to just watch and then you can have time for the exercises, yeah, when we do those. ESLint is really good at identifying patterns that are not the kind of patterns you want in your code base, and that includes stylistic issues, but also actual logical problems or domain specific things.

[00:01:02]
At PayPal we have several ESLint plugins where it's like, don't call this function with the request object because it's going to take that and try to pretty print it. And that takes a minute and our CPU spikes, this literally just happened and that release was a nightmare. And now we have a plugin for ESLint that means we will never do that again which is super cool, talk about practical.

[00:01:26]
So what we're gonna be doing is Is kind of some stylistic stuff. And I can show you a couple of examples of bigger plugins that prevent real, actual bugs from happening. And my hope is that when you go back to work or your own projects, that you can take the things about ASTs and ESLint that you've learned and prevent bugs in your program.

[00:01:51]
>> Student: It's almost pre-unit testing, right?
>> Kent C Dodds: Yeah, yeah exactly. And actually my workshop yesterday was about testing and your lines of defense against bugs are, first, static types. So use Flow or TypeScript, and if you can't or don't want to do that, then make ESLint plugins that guard against the common things that are in your code base or in your domain specific stuff.

[00:02:16]
And if that doesn't work, then you go into your testing. But if you just write test for every single thing you're gonna miss something, you're gonna have to write the same style of test everywhere. And so it's just much better to have something that works statically. I forgot to mention, I should have, ASTs don't run your code.

[00:02:40]
You'll notice there's no something to find in this file. So if I were running this is strict mode, this is gonna give me an error. But with the AST, it doesn't actually run my code and just analyzes it, generates this JSON or this JavaScript object for me. And so you can have this ESLint plugin that checks your code without even running it and you find out much quicker that you have a serious problem.

[00:03:07]
So the thing that we're gonna write here really quick, just to give you kind of an example of what the API looks like for a ESLint plugin, is something that will prevent you from doing this. So this is an if statement without a block. And the problem with this is, if you were to write it like this which is totally legit works, you might be tempted to say console.log something else and expect that it would run inside of the consequent for this if statement.

[00:03:40]
But it won't because it doesn't have a block. And so a pretty common pattern is just to always make sure you have a block. And then it's very clear whether or not your code is gonna be running in the block or not. So what you do with an ESLint plugin when you're developing it in the AST Explorer is I generally will put a couple of examples of the kind of thing that I want to lint.

[00:04:06]
So both the I'll have some invalid and some valid stuff. And I will make sure that my plugin can lint against the invalid stuff, but it doesn't break the valid stuff. And then once I have my plugin pretty solid in the ASTexplorer, then I can pull it down and I can write actual tests using these test cases.

[00:04:26]
And we'll do that.
>> Kent C Dodds: Okay, cool so on the bottom here we have our ESLint plugin definition. And what we're doing here is we export a module and that module is an object that has these two properties, meta and create. So the meta bit has some documentation information, has fixable and schema and we'll talk about those later, not right now.

[00:04:51]
And yeah, that's useful for generating docs and other things. And then it has this create method. And this is where you're creating what's called a visitor and both ESLint and Babel use what's called the visitor pattern. So let's talk about the visitor pattern really quick because you're gonna get to know it really well.

[00:05:14]
The visitor pattern makes it much easier for you to traverse your AST, because I tried to write a AST traverser just for fun and it was hard. So let's traverse this ourselves really quick. Let's say we wanna find all of the if statements. Here, and we'll make our thing a little bit more complex, just for illustrations purposes.

[00:05:44]
So we wrap this in a function. Our body has a function declaration. And so inside of this function declaration, every time we come across a function declaration, we have to know that a function declaration has a body. And that body can be a box statement, or it is a box statement.

[00:06:01]
And so now we have to know that block statements also have bodies. So as we're traversing things, we have to say okay I hit a block statement let me traverse down into the body of that block statement. Okay that has an if statement, perfect. I found what I was looking for but let's say inside of this if statement there's another.

[00:06:24]
>> Kent C Dodds: And now, once we hit this if statement, we have to know that there's a consequence and an alternate and we have to dive into that. And that can be a block statement. And so okay, we found a block statement, now we have to dive into the body.

[00:06:36]
So as you can see, with a bigger program, you have to know every single node type and where you can find more node types inside of that node. So it's not easy. I spent quite a bit of time on it and it was fun because ASTs are awesome.

[00:06:50]
But I'm so glad that we have these tools to traverse the AST for us because that's a lot of complexity we don't have to deal with. Any questions about traversing, how that, at a high level how that works? Pretty much just find the node. Know all the places that you can find more nodes in that node, and keep going until you find all the things you want.

[00:07:12]
So the visitor pattern makes it, I don't know why this red line keeps on appearing. Yeah, there we go. The visitor pattern allows you to say, I only care about nodes of this type. It's almost like a jQuery selector. So you say, I've got this huge DOM tree, just like an AST, abstract syntax tree.

[00:07:33]
It's the DOM. And I want to find nodes that have this class. With a visitor pattern, you don't have quite the granularity you do with css selectors. But you can, actually there's a cool project called esquery that lets you create an AST-like with css selectors, even pseudoselectors and stuff.

[00:07:57]
It's pretty wild. I just found out about it last night. It's kinda interesting. But with the regular visitor pattern, the only mechanism you have for selecting specific nodes is the node type. And so if I want to get all of the if statements, then I'm going to click on the if statement, find out what the node type is called.

[00:08:17]
And I will put a function on my visitor object that is of that name. And this function is going to accept a node, and now I can console.log that node, and write it in our browser here. We can pop open our DevTools, and we'll see two nodes were logged to the console.

[00:08:43]
So iconsole log and ast explorer all the time. It's really fantastic, it's the best way to develop. You can try and do like debugger statements and stuff and sometimes that's helpful, but lots of times it's kind of messy, so console logs are great. Okay cool, so we found two if statements, and we didn't have to do any traversing ourselves.

[00:09:06]
We just said, hey, I wanna find all the if statements, and it gave it to us. And it's important to know that this happens as ESLint is traversing the tree. So by the time it gets to that first if statement, it doesn't even know that there's another if statement yet.

[00:09:22]
Well, that's not true. It still does know that the if statement is there but some of the bindings from variables to where they're declared and that kind of thing, some of those things haven't been set up yet. And we'll run into some problems with that a little bit later.

[00:09:37]
Makes things a little bit more complicated but that's when things get kind of fun. We have our invalid first and our valid second so let's see some of the differences between these two nodes. So on our valid example, if we look at the consequent, that type is a block statement.

[00:10:01]
If we look at the consequent of our first if statement, the invalid example, that type is an expression statement. So that's the difference. That's the thing that we want to lint against. We wanna make sure that all if statements have a consequent that is a block statement, okay?

[00:10:18]
So let's go ahead and do that. What I like to do at the top of, most of my visitors at the top, I will filter out the things that are fine, that aren't what I'm looking for. So I'll say, if the node.consequent, actually, another little tip that I do is I'll console.log the thing that I'm trying to explore a little more about.

[00:10:42]
So I can see right here as I'm typing what it is that I'm gonna be testing. So node.consequent.type, okay so now I've got an ExpressionStatement and a BlockStatement. And I can say, is not equal to a BlockStatement or sorry, it's equal to a BlockStatement, okay. And then I'll turn that into an if statement.

[00:11:08]
>> Kent C Dodds: And I'll return. So now if I console.log the node, I should only get one, and it's the one that's breaking the rule. With this I need to tell ESLint, hey this thing broke the rules so do something about that. I don't have to like console log to tell the user, hey fix this.

[00:11:29]
ESLint has an API around reporting a problem on a node and then it takes care of actually telling the user that there's a problem based off of their configuration and stuff. So the way that we do that is with this context property or parameter that we get in the create function.

[00:11:49]
So the context actually gives us a lot of things, and one of those is the report function. And the report function takes an object, there are a couple of APIs but we're gonna use the object. And it takes a node property and a message property. And I'm gonna say, y u no block.

[00:12:14]
And if we look over here now, we're gonna see our ESLint rule is firing on the right thing.
>> Kent C Dodds: We're filtering out the ones that are cool, the ones that are not cool we report on. And ESLint will tell us there's a problem. So some takeaways here, ASTs are just JavaScript objects.

[00:12:40]
They are complex, there's a lot to them, but tools like ESLint and Babel will shield you from lots of that complexity, provide you a visitor pattern for you to just care about the things you care about. And then, because you don't have a whole lot of granularity at the top of my visitors, I normally eject early if it's not the type of or not the thing I'm looking for.

[00:13:11]
Okay.
>> Student: Question.
>> Kent C Dodds: Yes?
>> Student: So it's the actual transform that we're using that is the context, right? So you're saying the context is the insulin API.
>> Kent C Dodds: That's right, yep. Context is the ESLint API.
>> Kent C Dodds: And actually I should link you all to the docs for this stuff cuz you will probably be interested.

[00:13:37]
It's working with rules, and I'll just paste that in here. And there is the context object. This will tell you the properties you have available and the functions you have available to call. And we'll definitely be using that during the workshop.
>> Kent C Dodds: I'm gonna take this just one more step further and then we'll get into an exercise.

[00:14:05]
So it's not just the consequent that can be missing a block. You can also have an if false, whatever, or if thing. Then we will have a block here. So that's valid, but we can also have no block on our alternate console.log.
>> Kent C Dodds: And that's valid code, that's totally fine, but it isn't what we're looking for out of code.

[00:14:38]
And so here we're going to say, yeah, so we want both the consequent and the alternate to be a block statement, but right here, we don't have an alternate at all. And so we basically what we want to say is that the consequent and the alternate both if they exist that they are block statements.

[00:15:03]
So I'm gonna make a utility function. And I do this all the time when developing plugins. So right at the bottom, I'll make a isBlock function here that takes any node, and it will return, if there's not a node, then that's fine. It's basically, maybe I should name this function is block or doesn't exist, but that's kind of long, so I'm just going to leave it this way.

[00:15:31]
So if it's not, if the node doesn't exist then that's fine. Otherwise, the node.type needs to be a block statement. So then I can swap this with is block.
>> Kent C Dodds: And that should still be working and I can in addition to that, I can say and the alternate.

[00:15:55]
So now I'm getting a warning except here's another little tip is where the warning appears is on this if statement. But the consequent has a block, so that's fine. And so it's a little misleading, so what we can do is instead of reporting on this node, we can actually report on the node that actually is the problem case.

[00:16:22]
And so we could rework this a little bit further if we wanted to, to basically say if it's the consequent that's the problem, then report on the consequent. That's what we'll do, we'll say if it's not consequent then we'll report on node.consequent. Otherwise, else if (!isBlock(node.alternate)), then we'll report on the alternate.

[00:17:01]
Sorry, I'm used to prettier formatting all my code for me. Cool, so now we're getting an error message like very specifically where the problem is. And the error message should probably be a little bit more friendly and useful but we're having fun so