Introducing Abstract Syntax Trees

[00:00:00]
>> Kent C. Dodds: All of this talk about is really awesome things, gets me super excited. But, what is an AST? We talk about how this AST can accomplish these really cool things, but what even is that? So this is code, it is a module, it is something you can actually install to code transformers' names, and it'll get you a random Transformers' name.

[00:00:25]
I don't even know what their names are anymore. But if you're into Transformers you might care. So this is code, and it's code for humans. I'm using variable names here that are useful for us, but as far as the computer's concerned that could be variable A and this could be variable B.

[00:00:43]
It really doesn't matter. But for us it does, and so we write our code in a way that, that makes sense for us and then, we send that off to a computer and it turns it into something that looks like this. This is a JavaScript object that represents that program.

[00:01:02]
It's much bigger to represent what we had there. It's pretty wild stuff. So this is an AST, maybe it looks a little bit scary, maybe not, but hopefully by the end of this, it's not scary to you and it's just a JavaScript object cuz that's really all it is.

[00:01:22]
It's pretty cool. So that's kinda hard to read so we're going to explore an AST using this really neat demo from this company called JointJS. This demo is, actually, it's purpose is to show you how cool their draphing tool is, and they just happen to be using an AST to demonstrate that, so we're gonna be abusing this a little bit.

[00:01:52]
Thank you JointJS. Okay, cool. So this is some code, here let's bump this up a little bit. And we're declaring some variables. We have this function declaration, and we're making some assignments here. If we click Show AST, then we'll see that AST out here. So let's explore this a little bit.

[00:02:14]
The AST starts out with a program. And the program has a body, and all of what's inside of our program is in that body. And then as part of that body we have four things. And that first one if our variable declaration. So you can see at the top of our program is that variable declaration that's happening right there.

[00:02:36]
And that variable declaration has a single declarator. So the variable declaration of type var has a declarator that is a = 42. And that variable declarator has a left side and a right side. The left side is a and the right side is 42, literal 42. One of the things that I really like about working with ASTs is it makes me better at JavaScript.

[00:03:00]
Because I understand what the purpose of these different pieces of the language are. And also how they work together. So sometimes I forget that a variable declaration can have multiple declarators, and that's why these two things are separate. So if I add another declarator here, we'll say q = 23, we'll show the AST again.

[00:03:27]
Now, this variable declaration has two variable declarators. And each have a left side and a right side. Left side and right side can be a little bit more complex, so we could say a + 23. And now, this q variable declarator has a left side of the identifier q.

[00:03:48]
So this is what we're assigning things to. And then the right side is a binary expression. And so now, like, we see this addition operator. And we can actually give it a name. It's a binary expression. And a binary expression also has a left side and a right side.

[00:04:03]
And it can go down as deep as you want. You can keep going forever and ever and ever. And that's just JavaScript. But it kind of changes the way that our ast represents itself, I think it's pretty rad. So the left side of a BinaryExpression can be another one, and another, and another.

[00:04:27]
And what's cool about this is you get to learn about precedence, about how JavaScript is evaluating things. So if we toss in a multiplication in here, it's gonna throw the AST off like it's gonna be totally different. So now this BinaryExpression branches off with the right side as a BinaryExpression 23 times a because of precedence.

[00:04:52]
And so you learn about how JavaScript works by working with these ASTs. We'll just explore a couple more things here just so you can get some exposure to a few more node types. And I failed to mentioned each of these is called a node in AST, the Abstract Syntax Tree.

[00:05:11]
And each one of these can have branches, and then you get to your leaf nodes. Often, that winds up being an identifier or something. So here we have another variable declaration, that's not interesting anymore, we know that. Now we have a function declaration, that has a block statement.

[00:05:32]
And that block statement has a body. Which can have one or more elements inside of it. It only has one, it's a return statement. Let's make it have another one. So I'll say d += 1. And so now that function block statement has two items in it. So the first is an expression statement.

[00:05:55]
Within an assignment expression, with the left side as d and the right side as 1. And then our block statement also has a return statement as the last item, and that's a binary expression for the left and right. So we could do this all day long. There are all kinds of different node types that you can mess with.

[00:06:14]
But it's more fun to actually really mess with them and we're gonna do that next. Before I move on, any questions about AST's in general? Yeah?
>> Speaker 2: There's a question online, does the variable declaration distinguish between LAR CONST or LET. And how is that represented in the tree?

[00:06:37]
>> Kent C. Dodds: Yes, it does, I will show you in just a sec. That's a great leading in question for this bit. So everybody feel free to go to ASTExplorer.net. And you won't see what I'm seeing, unless you're on my slides, you click on this link. But, yeah, it's fun to play around with regardless of what code's in there.

[00:07:02]
So yeah, let's, actually, I'll show you this variable declaration right here. And the distinction is in this kind property on the variable declaration node. So you can also, here we'll change this to var, that changes the kind to var. And let that changes the kind to let. So before I explore this code and the ast any further, I wanna just show you the AST Explorer.

[00:07:35]
Because as you work with ASTs, you'll spend a huge amount of your time in the AST Explorer. So here you can Fork, and Save, and create a new one, share things. And then, I don't even know what this icon is supposed to be, but this all of the different ASTs that are available to be parsed.

[00:07:58]
So you can actually parse the CSS AST, or the CSS code into an AST and graph UL and mark down and like actually, wait, there is no mark down but there should be and you could add it because it's awesome, because there are markdown parsers that create ASTs out of things.

[00:08:20]
Yeah, JSON, I don't understand why you'd wanna parse JSON, to be honest. From a JSON object into a JSON, I don't know, but I'm sure some people have use cases for that. Even regex, all kinds of really interesting and awesome things. And I'm glad that we could do this, cuz it's a lot of fun.

[00:08:41]
We're gonna stick with JavaScript here. And then this next session, based off of the language that you've chosen, you have various parsers, and with JavaScript, there are several parsers. And there's an effort to standardize on the AST that they create, but they do create slightly different representations of the code.

[00:09:07]
So here we can see if I choose acorn, the root node is a program. If I choose babel-eslint, then here the root note is a program, as well. But if I choose Babylon, the root note is a file. So there are like some subtle differences between these and each has like some extra properties that are useful for the purpose of this parser.

[00:09:34]
So we're gonna be starting out with ES lint and ES lint's parser is espree. Or, you can use babel ESLint which uses babel's parser and then converts it into ESLint supported AST. And so you'd use that if you're using features that the ESLint parser doesn't support yet. But most things are supported by Esprit, so we'll stick with that.

[00:10:03]
You can specify some options here, and then you can do this really neat transform thing right in the browser. You can write your plugins in the browser. This is the enabling tool for writing plugins and for pretty much everything. It's awesome. As I click around, I have this autofocus thing checked and so as I click around it's going to automatically focus to the node that my cursor is on.

[00:10:34]
So you'll see that most things that I click on are identifiers. So those are bits of text that identify a piece of a node, that's the idea of this property. And so if I go ahead and explore a little bit further, I'll just, here we go. We're on a block statement in this random function.

[00:11:00]
And we can see some properties here. It gives us a range, a start, and an end for where the first character exists in the source code. It gives us a body and that body is an array and it only has one node it's just an if statement and inside of that if statement it has a test and a consequent and an alternate.

[00:11:23]
Now we have like names for these things, like you always use to say, the if statement like passes and then the other one. Now you can say, okay, my test is failing, so my consequent isn't running, my alternate is running. You can actually talk about code using proper names, which is cool.

[00:11:44]
So the test is a binary expression. Binary expression nodes have a left and right and consequent, which in this case has a body but it doesn't have to have a body. We can actually fun fact with JavaScript. You don't need to have those braces. And you learn stuff like that when you're messing around with ASTs and you realize that would be hard code to read and update and stuff.

[00:12:09]
So we should always do blind statement. But you don't have to. That consequent can be any expression. And then we have an alternate and it's a BlockStatement that has a body. So, again, we could explore ASTs like all day. There are tons and tons of different node types, but we're going to actually dig into this and manipulate and inspect these things to get a good idea of how to actually leverage this to be useful for the things that we're doing for work or whatever.