Deep JavaScript Foundations Module Patterns

Transcript from the "Module Patterns" Lesson

[00:00:00]
>> Kyle Simpson: Our final topic in this discussion unit on scope and closure is to look at the module pattern. Specifically, we wanna take this out of the realm of esoteric academic ideas to something practical that we can repeat and use in our everyday coding. The module pattern is probably the most important of all the code organization patterns in programming, especially in JavaScript.

[00:00:23] [COUGH] So, I wanna start first by talking about what's not a module. That way, we fully understand and appreciate what is a module. Here, I've created a foo that points at an object that has some stuff on it. In this case, an o variable and the bar method.

[00:00:37] And it is true that I have collected some variables and some methods into a single unit, but that doesn't really satisfy the spirit of what encapsulation is about and encapsulation is what is necessary to fully realize the module pattern. Encapsulation means, specifically hiding of information that is not necessary to be seen to the outside world.

[00:01:03] We talked earlier in the course about the principle of least privilege or the principle of least exposure. We talked about that with block scoping like we hide a variable from an outer scope, if it's not needed. The same principle applies at the API level. If a method or a variable isn't needed, you don't wanna expose it to the outside world that is the rest of the code if you don't need to.

[00:01:26] If you expose it, it means that somebody may take advantage of and use that method in a way that is different than you were expecting. Now, the way this typically plays out is you have some undocumented method. A private method that people weren't supposed to use and maybe you make it like with an underscore in the variable name and you tell people, don't use it.

[00:01:46] And yet, somebody starts using it. Somebody starts relying upon that method. And later, you wanna go back and refactor and change that method or remove it and you can't. Because now, you're gonna break people's code. You're like this was undocumented. You're not supposed to use it and the real response is you were supposed to use encapsulation to prevent people from using it.

[00:02:08] That's what that design pattern is for. So in this case, we don't have any way to prevent somebody from using o or bar. At best, we might have the obscurity of putting an underscore in the name. But real encapsulation would take advantage of functions scope to hide the things that aren't necessary.

[00:02:25] So bar needs to be exposed, great. But what if we don't need to expose the o variable, because that's an internal detail that's not necessary to the outside world. So, let's look at using function scope and in particular closure to realize the module pattern. This is the classic module pattern, codified most notably by Doug Crockford back in the early 2000s.

[00:02:49] You'll notice that I have an outer enclosing function. In this case, it's actually an IIFE, although it doesn't have to be. An outer enclosing function that starts on line 1 and then executes itself down on line 11. Inside the function, when I declare a variable or if I were to make a function declaration.

[00:03:05] By default, all of those are lexically hidden scope-wise from the rest of the outside world. The only stuff that's gonna be available for you to see from the outside world is whatever I put on this object right here and that object I return, which gets pointed to by the foo variable represents the publicAPI for this module.

[00:03:26] It is the stuff that we choose to expose and anything we don't expose stays hidden. So you can't say something like foo.0, because 0 is lexically hidden inside the scope of that function. You can, however, say, foo.bar. Because you choose to make that public on the API. That's what we call the module pattern.

[00:03:46] Technically, we're not really abusing, but sort of reusing some existing mechanism. In this case, function and closer to implement this pattern. It's not exactly like having a module keyword that we can use. We'll talk about syntactic support for the module pattern in just a little bit, but it's worked well enough that this has been, again, as I said, the predominant pattern for organizing our code across pretty much all of JavaScript for a really long time.

[00:04:14] The other competing pattern of course is classes and we will talk a little bit later about classes. Before I move on from talking about this specific slide, I just wanna emphasize that while this is a syntactic pattern. There's a layer of thinking that you need to begin to adopt and the exercise we're about to start working on will encourage you to think more deeply about this particular or will encourage you to challenge your way of thinking about this stuff differently, because the real questions you wanna ask yourself are not just does my application do the right thing.

[00:04:49] Think back to the exercise code we've been working on running through this workshop. All the functions are just hanging out there out in the open global space. A, that's not great organization that all that stuff is hanging out in global space. But B, it's not good organization. Cuz it's not clear where the appropriate separation or the appropriate boundaries between functionality are.

[00:05:12] I mean, you can think about it as simply as there's one set of responsibility which is managing what happens in the DOM that you iced off and there's another set of responsibility for managing the data model. That in of itself is two separate pieces of functionality. The way the code is currently constructed, that stuff is all conflated stuck together.

[00:05:33] And it's difficult to tease out which parts are UI, which parts are data model. So the module pattern is not just about it's pretty or it like organizes things into a publicAPI. It's really about forcing you to think about how to design your software, so that the proper separation of responsibilities is in place and you're gonna be wrestling with that multiple times.

[00:05:58] There is three parts to the upcoming exercise. You're gonna get to wrestle with that question multiple times in this next exercise. So, this is the classic module pattern. Now [COUGH] there is one thing to observe about this which may not be obvious at first and that is the object that we return, the one we're returning on line five.

[00:06:17] It's sort of an anonymous object, because we return it and foo can reference that object. But we don't retain any internal reference to our own publicAPI. Inside of our module, we don't have any access to our own publicAPI. And I don't know if that sounds strange to anybody else, but that feels really awkward to me that a module doesn't have access to its own publicAPI.

[00:06:42] Why might you need that? Well, a couple of use cases. One, I might wanna call one of my own publicAPI methods from internally. Two, I might wanna add of remove properties or update property values or add or remove methods from my own publicAPI. If I don't retain an internal access to it, I can't change my own API.

[00:07:03] I can't do anything with my own API and then there's the stylistic question which is if I were to try to reference something, it would not be obvious that I was dealing with my own publicAPI.
>> Kyle Simpson: So this next slide shows a very slight variation of this pattern, a modified variation of the classic module pattern where I actually create an object that serves as the publicAPI.

[00:07:28] But I save it to an internal variable called publicAPI and then return that reference. Same principle applies, this is still a scope closure based module pattern with encapsulation. But now, I have an access to publicAPI. So when foo points at that object, we now have foo on the outside pointing at the object.

[00:07:48] And on the inside, publicAPI pointing at the same object. And now, I can use that reference internally. I can call my own methods. Aadd and remove methods. Update property values as I see fit. Even if I don't need to do that, I've just gotten into the habit of always declaring my module pattern by explicitly calling out my publicAPI.

[00:08:10] Because stylistically, it makes it clearer that's exactly what this object is doing. It is serving the purpose of being a publicAPI. Put this in your notes cuz it's not in the slide but it's important for you to know. There are two characteristics implicit in this definition of modules and both of these characteristics need to be present for you to legitimately be using the module pattern.

[00:08:34] The first characteristic is that there must an outer and closing function that runs at least once. And I would even more generalize it to say there must be an outer enclosing scope. That's almost always gonna be a function but technically it doesn't have to be a function. But there has to be an enclosing scope.

[00:08:52] In a couple of slides ago, we didn't have enclosing scope so that's why it failed the module definition. Here, we have an enclosing scope. This function happens to run exactly once. We have a special term for things that only get created once and it's called singleton. So this is kind of like a singleton module.

[00:09:11] There's only one instance of it. Had I made that a regular function that could be called multiple times, every time that function was called, it would produce a new instance of the module. So it's kind of like a module factory function. So if it wasn't an IIFE, if I took off the parenthesis and just had a function that I could call multiple times, that's a module factory.

[00:09:35] But that first characteristic, we need to have one outer enclosing in this case, function that runs at least once. Second characteristic, there needs to be at least one internal function that returns back in the publicAPI that has closure over the internal state. If you're not taking advantage of the internal state of this closure, then you're really missing the whole point of the module pattern.

[00:10:01] Why go to all this trouble to create the wrapper syntax. The boiler plate overhead, if your not gonna take advantage of being able to maintain that state overtime hidden inside of the encapsulated scope. So really, it's those two characteristics have to play together for there legitimately be a module pattern.

[00:10:21] Outer and closing function that runs at least once, inner function with closure of that scope that gets return. Beyond that there are lots of variations that you can use. Some of you may have heard of AMD style loaders like RequireJS was a very popular one for many years.

[00:10:42] They typically look something like what I'm showing here whether some function like define or something like that. And you give it a function that's suppose to return back your publicAPI for your module and you might optionally tell it, this is what I wanna call it. This is the name of my module, and I just wanted to point out that this is not some magical secret sauce.

[00:11:03] This is exactly what I just showed on the previous slide. The only part that is hidden here is, you don't see that function being executed and the return value being assigned to a variable called foo. That's the only part that this loader this define loader is actually doing for you but everything else behaves by exactly the same rules.

[00:11:24] We have internal hidden variables or functions that aren't by default accessible. We make them accessible by adding them to the publicAPI, so we end up with a foo module that has this bar method on it. So I referred a moment ago to, this is kind of repurposing function syntax to create this module idiom out of thin air.

[00:11:50] But it's not really exactly dedicated syntax. Wouldn't it be nice if there was dedicated syntax? As of ES6, we got dedicated syntax for it. As via 6 we have import and export that allow us to express that a module by default has everything hidden. And export let us tell it what we wanna add to the public exports list, import says, we get to list what we wanna import from a module.

[00:12:22] So if I had a file called food.js here, and I had a hidden internal variable called var0, you'll notice that it's just by default hidden. I didn't need to wrap anything around it because the entire contents of the file are treated as the module. That is something very unique and different compared to the way we've been thinking about modules in JavaScript.

[00:12:44] The file is the module, the file contents. The way it gets treated as the module contents is based upon how that file gets loaded. If it gets loaded by an import statement like we see in the second code snippet, then it's gonna get interpreted as a module and everything will be hidden within a module scope.

[00:13:05] The only stuff that'll be accessible to the outside world is the stuff that we choose to put export in front of. So what does that mean to be file based? The most pragmatic or practical observation that you'll then make is, it is impossible for you to package multiple modules into the same file.

[00:13:26] The entire contents of a file will be treated as a single module. So, if you have 100 separate modules for your application, you're gonna have to put those into 100 separate files. And you're gonna have to keep those hundred separate files completely separate from the perspective of how they load, even into the browser.

[00:13:51]
>> Kyle Simpson: There's a good chance that that might throw up some alarm bells in your head because you've also probably heard for 15 or 20 years of the web's history. Don't load hundreds of separate files, cuz the performance will be terrible. Concatenate everything into one file. So it might seem like this design is antithetical to web performance.

[00:14:11] The perspective that TC39 used was that by the time module support had rolled down in full scale to all of the web browsers. Simultaneous with that our concurrent with that, we would had to roll out off another technology assistive technology if you will which is HTTP version two.

[00:14:33] And the primary observation that's different between version one and version two of HTTP, is that in version two, everything is delivered over a single persistent socket connection. Not terribly dissimilar from web sockets. There's one connection made between client and server. That connection stays alive for the entire lifetime.

[00:14:56] And all the resources that need to go out, go out over that single connection. As a matter of fact, they can be interweaved together so the worst possible thing you can do an HTTP version two, is package everything up into one file. And send all of those bytes serially down the line.

[00:15:13] The best thing you can do, is have lots of separate resources and send them concurrently.
>> Kyle Simpson: So their perception or their forethought was, this won't be a problem to be file based because by the time people adopt ES6 modules, they'll also all likely be adopting HTTP version two.

[00:15:33] But that means for you today is, don't go adopting ES6 modules until you're also ready to adopt HTTP version two. Version two of HTTP does require a more intelligent server. You have things that the server has to figure out like server push, where it receives a single request from the browser and the server knows what resources are gonna be asked for.

[00:15:55] So, it pushes them up preemptively and all kinds of other stuff. You're gonna need a more sophisticated server to figure that stuff out. They exist, but you're gonna need that, it's not just gonna automatically come right out of the box.
>> Kyle Simpson: So that's the first major observation, modules are file based.

[00:16:12] And that will change a bit how you might design your application. The other thing to know, is that modules are by default singletons. There's only one instance of the module in ES6. If I called import bar, like I do there on the line, one with the second snippet.

[00:16:28] If I call that import bar 15 times, I'm not gonna recreate the foo module 15 times. It only gets created and run once. And then I get 15 separate reference copies of the same module instance.
>> Kyle Simpson: So if you're in the habit of liking to have different module instances like you think of them as module factories when you require them or something, you're gonna have to implement that layer of module factory on top of your API.

[00:16:57] Have to export a method like create or factory or something, that can generate those. Because just importing it will not create a new instance. That's another thing to be aware of. The biggest caveat that exists here is that there's a big incompatibility between the syntax that rolled into JavaScript.

[00:17:16] And the most predominant environment where we have modules today, which is the npm ecosystem in Node.js. To date, there's 420 something thousand public packages, plus who knows how many millions of private packages that have been written and published to the npm repository. And 100% of those are incompatible with this new ES6 syntax.

[00:17:40] Moreover, Node.js itself is incompatible with this new syntax because Node.js needs to support, not only the old style, required style, but the new import style. And while it seems like that would just be a simple matching up of stuff. It turns out when Node went to go implement this, they're like crap, this doesn't work, and we can't break our existing ecosystem to make it work.

[00:18:04] So then they had to go back to the drawing board. They went back to JavaScript, the TC 39 committee and they're like, sorry, this is awkward but that's not gonna work. And so they've been hashing this out now for the better part of a year. Most recently within about a month or so they announced that they think they have a path forward for compat, but that it will take at least another year to iron all of that out.

[00:18:30] So it's not gonna be any time soon that you can rely upon this, even though it's in the [LAUGH] specs standard, that you can rely upon it. It may change, they're talking about requiring you, if you're gonna do node modules that are ES6, you're gonna have to use a special file extension or some other kind of ridiculousness.

[00:18:49] This stuff is still in flux. So my recommendation is, while it's cool to look at, and I'm excited about this syntax eventually. For now, I've chosen to hold off converting to this module syntax. For now I continue to use the existing classic module pattern. We call that common JS, we call it UMD, we call it AMD, whatever terminology we put on top of it.

[00:19:13] That's how I ship my modules so that they run in both Node and in the browser. Some people have started using this as an authoring format, but then they transpile from this format back to something like common JS. Personally for me, I think it's the reverse. Personally for me, I'm gonna continue to author for a long time, in the existing format and I eventually might start transpiling the ES6 modules.

[00:19:42] So it will become instead of an authoring format, a target format for the environments that support it. So personally, for me, I'm not likely gonna start offering in this for maybe at least another five years or so. Let all of this stuff iron itself out. That's my own take.

[00:19:58] You can make your own decisions as makes sense for your team.