Building Your Own Programming Language Parser Generators
Transcript from the "Parser Generators" Lesson
>> Steve Kinney: I do wanna talk about one other thing, which is, there's a whole class of things called parser generators. And these are effectively, it's kind of meta. It's a domain specific language, or there's various ones of domain-specific languages for generating domain-specific languages, right? And I am of mixed feelings on them, one, in learning them, you're kind of learning that parser generator.
[00:00:30] Some of them are based on like Bison or Yak, or something along those lines. But generally speaking, you're learning that kind of parser generator, and not any of the stuff that goes on behind it. But there's a trade-off and so here's a little, the bold part is the important one.
[00:01:15] We don't, in fact, do that, but that would be the hope in this case. But generally speaking, I'll show you a few of them, if they're interesting to you, I think they're worth taking a look. But they have never effectively solved the problem for me. And it's not that I'm writing them off, I see promise and merit in them.
[00:01:34] But I think it's always worth evaluating the trade-offs of a thing that you have kinda the complete flexibility and control over, versus something where you're kind of bending to the way it does things. So you see, this is even, to kind of do a very kind of simple syntax, you're basically writing a bunch of all the rules for a language in a very linguistic sense.
[00:02:21] But then also, there's some additional stuff along the way. So it is another area, if this is a thing you're really into, there are things like Antler, which is kind of Java-based, that do some really kind of impressive stuff as well. Worth looking into, but I think for me, I've a lot of times got a lot of benefit of just a piece or two of some of these concepts and being able to use them.
[00:02:45] So there's Jison and here's, this one's I think Chevron, where you kind of define everything. You get these kind of cool graphs, but there's a learning curve that you have to kinda make the choice on. Nearley is another one, you can kind of see, their syntaxes [INAUDIBLE]. This one will match a hexadecimal color and CSS, or different colors, right?
[00:03:11] I would argue that we have the skill set now to write a parser to do that too, right? And so you kinda have to make the choice in those cases, or you don't have to make the choice, right? And there's some other list-specific things that we didn't get into.
[00:03:24] One of the things I said during one of the breaks was, a lot of times, when implementing a list, it'll be a series of arrays. Ours was that we also kind of had one foot in the tree realm. And the reason for that was, I think there's a lot of value in understanding the visitor pattern on the ASTs and transpiling to Babel.
[00:03:46] And that kind of got a little hairy and complicated with a bunch of arrays and casting a bunch of stuff. There's a bunch of really cool stuff that you can do. So this is creating a lambda, which you see we could add to the environment. I think what's kind of cool here is, you basically are making a function.
[00:04:00] And you're using those parameters and you're kind of merging them with the environments. You're creating like ideas of scope, right, by having, okay, whatever's in this scope, if there's an x in our local scope, it overrides the global scope. But then when we're back out of that one, we're using the global scope again.
[00:04:33] But I think that as you begin to think about different concepts in languages, it does also help to help you understand how they work when you start to implement them as well. So on top of the very practical things, I think there's also a lot of rich language we use.