API Design in Node.js (using Express & Mongo)
This course has been updated! We now recommend you take the API Design in Node.js, v4 course.
Table of Contents
Nodejs RefresherScott Moss begins the course with a brief refresher on Nodejs. He talks about the various ways it can be used and explains Node’s package manager, NPM.
CommonJSNode uses CommonJS as it’s module loader. Scott explains how to use the require() method to load built-in modules, Node modules, and user-created modules. He also talks about the differences between using exports and module.exports when creating a module.
ExpressWhile Node has a built-in server module, using it to create a basic server can be overwhelming. Instead, Scott will be using Express to handle server requests. Express is one of many server frameworks for Node which greatly simplifies the process of creating your own API.
Using ExpressScott walks through a simple Express example. This example is a basic HTTP server listening on port 3000. The server has a couple routes for handling both GET and POST requests for a /todos API.
Exercise 1In this exercise, you will create a basic web server with Express. The server will send back the index.html file when it receives a ‘/‘ request. It will also send back JSON data when a /data request is received.
Exercise 1 SolutionScott walks through the solution to exercise 1. He also shares a few workflow tips like using nodemon to auto-restart the server after each change.
Getting RESTfulAn API that’s considered to be RESTful should be stateless, use HTTP verbs, and expose a directory-like URL pattern for routes. While REST is not a standard, it’s how most of the modern web APIs are built.
Anatomy of a REST APIScott discusses how to create a REST API. He begins with a model of the data. Then he makes the routes to create, read, update and delete (CRUD) the data. Once these resources are in place, the API can be created.
Exercise 2In this exercise, you will create all the necessary routes for the ‘lions’ API. Before beginning the exercise, Scott explains the middleware used to serve the client application.
Exercise 2 SolutionScott walks through the solution to exercise 2. He begins with the GET and POST routes which handle the retrieval and adding of lions.
Exercise 2 Solution, continuedScott finishes his explanation of the second exercise by covering the PUT and DELETE routes which update and remove lions.
What is MiddlewareMiddleware is a function that has access to the request object, response object, and the next function which points to the next middleware. Since the request and response object available, middleware is able to change these objects, run any code, or even end the request/response cycle.
Types of MiddlewareIn Express 4.x, there are five different types of middleware: 3rd Party, Router, Application, Error-handling, and Built-in. Scott explains a few of these types and shares a few code examples of their use. He also spends some time talking in-depth about error handling.
Middleware ExamplesThere are many ways mIddleware can be used within an application. Scott shares a code example that demonstrates a few of these ways. He also compares 3rd party middleware with middleware created within the application.
Exercise 3In this exercise, you will add a number of middleware methods to the server. The middleware will handle incoming lion requests, errors, and additional routing.
Exercise 3 SolutionScott goes through the solution to exercise 3.
RoutersExpress 4.x allows for more than one router in an application. This means developers can use a router in a more modular way. A router can have it’s own routing, stack, and functionality. Scott demonstrates this by showing a code snippet with an encapsulated todosRouter.
Exercise 4In this exercise, you will make a new Router for the tigers module. You will then mount that router in the application alongside the lions module.
Exercise 4 SolutionScott walks through the solution to exercise 4.
Error HandlingBefore moving on, Scott makes a few code optimizations to demonstrate who to simplify the routing syntax. He also talks briefly about error handling. Node doesn’t output any error details by default so Scott adds an error handler and explains how to use the Node debugger/inspector.
Audience Q&AScott spends some time answering audience questions. These questions range from other debugging tools, routing tips, and logging.
Audience Q&A continuedScott continues his audience Q&A with a demonstration on how to use ES2015 features with Node.
Testing in NodeUnit testing in Node is very similar to unit testing in the browser except for the absence of the DOM. To make the testing process easier, Scott recommends exporting the main application so it can be used by the testing framework. He also covers a couple strategies for writing API tests and saving development dependencies.
Node Environment VariablesScott spends a couple minutes talking about Node environment variables. All environment variables are stored on a global process.env variable. One important variable is the process.env.NODE_ENV which determines if an application is running in a production or test environment.
Exercise 5In this exercise, you will write tests for all the CRUD routes in the application. You will use Mocha to run the tests and view the results.
Exercise 5 SolutionScott demonstrates his solution to exercise 5. He also shares a few variations of the tests since there are many ways they could be written.
Testing Q&ABefore moving on, Scott answers a number of audience questions on testing strategies and demonstrates a few alternative approaches.
Organization & Configuration
Application OrganizationScott talks about the component approach he uses when organizing application files. Within a component, models describe how the component’s resources will look, controllers access these resources, and routes help define how the API will be exposed.
ConfigurationOne important role of the configuration file in an application is to determine the environment in which the application is running. Scott explains how using the process.env.NODE_ENV allows him to set specific environment URLs, application secrets, or logging settings.
Exercise 6In this exercise, you will be loading the envConfig variable with the correct configuration data based on the environment. You will also be properly configuring the routers. Before beginning the exercise, Scott briefly walks through the application architecture.
Exercise 6 SolutionScott walks through the solution to exercise 6.
Mongo IntroductionMongo is a NoSQL document store. Data does not have to be modeled in advance with Mongo which makes it very easy to setup and use. Scott demonstrates how to create a new collection and insert records using the Mongo command line interface.
Exercise 7In this exercise, you will connect to a Mongo database. Once the database connection code is in place, you can verify the connection through the router.
Exercise 7 SolutionScott walks through the solution to exercise 7. He also briefly discusses collection naming conventions in Mongoose.
SchemasMongoose schemas can be used to add structure and validation through a process called data modeling. Scott walks through the code to create a schema and demonstrates the different ways properties can be defined.
Schema TypesScott walks through a more complex schema definition to introduce all the different data types that can be used when defining properties. These data types include Boolean, String, Number, Date, and Buffer. Scott also talks about nested objects and additional validation rules like min and max.
ObjectIdThe ObjectId type is used to create primary/foreign key relationships between objects in the database. While this property looks like a String, it’s actually an Object with a unique reference. Scott explains how to use the ObjectId type and answers a number of audience questions about it.
Blog Schema RepresentationScott spends a few minutes looking at a JSON representation of the schema for the blog application. After walking through the properties, he answer a few questions about migrating data to a new schema.
Exercise 8In this exercise, you will design schemas for the category, post, and user models in the application.
Exercise 8 SolutionScott walks through the solution to exercise 8.
Querying Data with MongooseMongo has a sophisticated querying language with a lot of options. The find() method, for example, is passed a query object. While there are variations of the find() method, most method calls look similar. A callback is passed as a second parameter. Scott demonstrates how to use the callback to step through, update, and delete documents.
PopulationsSince Mongo is a NoSQL database, there are no join tables. Relational data instead uses Populations. Populations behave like a join table, but are used at call-time.
Exercise 9In this exercise, you will build out the controller files for each module.
Exercise 9 SolutionScott begins the walk through for exercise 9. First he starts with the params method in the categoryController.
Exercise 9 Solution, continuedScott continues the exercise 9 solution by explaining the get method in the postController and the getOne method in the userController.
Consuming PromisesUsing Node’s file system API, Scott demonstrates how to consume and Node-style callback as a Promise. He wraps the API call in a native Promise and returns a call to the resolve or reject methods.
Nested PromisesAs promises get more and more complex, additional methods can be used to make their consumption cleaner and more readable. Scott explains how to error handle individual promise calls. He also introduces the all() method which will consume an array of promises.
JSON Web TokensThis application will use JSON Web Tokes (JWT) to handle authentication. JWT will replace the traditional cookie/session authentication used in traditional web applications. Scott introduces JWT and talks about how they will be used.
Using JWTScott shares a brief code snippet demonstrating how to work with JWT. A token is generated by hashing some identifying user data with a secret key. During subsequent requests the server will use the same secret to verify the request’s token is valid.
Usernames & PasswordsScott begins explaining the process of creating a new user. Users will supply their username and a password. A hashed version of the password will be stored in the database. When a users logs on in the future, the submitted password will be hashed and compared to the stored password in the database.
Authentication with MiddlewareMongoose allows new methods to be added to the Models and Documents in the application. Scott explains how these new methods will allow a middleware API to be built around the authentication process by using lifecycle events like “before save” and “before validation”.
Exercise 10In this exercise, you will connect the routes with their controller methods.
Exercise 10 SolutionScott walks through the solution to exercise 10. He also demonstrates how to create utility functions for generating the routes.
Executing CRUD OperationsBefore moving on to authentication, Scott spends a few minutes testing the current codebase by executing a few CRUD operations from the command line. He adds a new user, creates a category, and creates a blog post. He also explains how populations work.
Authentication ConfigurationBefore moving into the next exercise, Scott gives a quick overview of the configuration changes he made to integrate the authentication module. He explains the need for a couple utility functions as well.
Exercise 11In this exercise, you will implement the getFreshUser and verifyUser methods in the “auth” module. You will also implement a sign-in method that will generate a token to be sent back to the client.
Exercise 11 SolutionScott explains the solution to exercise 11.
Testing the AuthenticationWith the authentication now fully implemented, Scott spends a few minutes testing the API with one of the seeded users in the database. He also answers a few questions about writing a custom authentication layer versus using other third-party libraries.
Identifying Sensitive RoutesNot all routes in an API should be publicly available. Scott goes through the post, category, and user modules to identify routes which require authentication. He also explains edge cases where a route should only be available to a single user versus all users.
Understanding CORSScott installs the blog user interface which will be used throughout the rest of the course for working with the API. Since the blog interface runs on a separate port, CORS will need to be enabled to allow access to resources form different origins.
Testing the UINow that CORS has been added, Scott spends a few minutes testing the new user interface. He demonstrates the admin and sign-up functionality and explains what routes still need to be secured.
Exercise 12In this exercise, you will secure the routes that create a new blog post. This way, only logged in users will be able to create posts. You will also need to enable CORS so the UI application can communicate with the API.
Exercise 12 SolutionAfter answering a few audience questions, Scott walks through the solution to exercise 12.
Exercise 12 Solution, continuedScott continues talking about the solution to exercise 12. He demonstrates the working user interface and uses the Chrome Developer Tools to show the authorization tokens in action.
Deployment OverviewWhile deployment of a Node application is relatively easy, there are some things to consider. Scott runs through some of his recommendations like using ENVs for secrets and ensuring the platform has access to all necessary build tools.
Deploying to HerokuScott will be deploying the blog application to Heroku. He logs into his Heroku account and creates a new application. Scott then demonstrates the process of pushing the application to the deployment server and debugging any issues.
Configuring the DeploymentThe Heroku deployment server is missing some of the necessary environment variables for the application. Scott demonstrates how to configure the server for the correct environment and database URI.
Q&A Part 1Scott begins the final Q&A by answering questions about when to use query string variables, how to identify quality node libraries, and the scalability of Mongo.
Q&A Part 2Scott continues the final Q&A with some ways to scale the MEAN stack, recommended resources for beginners, and NPM modules for interfacing with other databases.
Q&A Part 3Scott wraps up the course with a few last Q&A questions about service-oriented architectures, using different third-party services, recommendations for style guides or code-quality checkers, and profiling/debugging Node.