API Design in Node.js (using Express & Mongo)// Dec 30, 2015

Table of Contents

• Node.js

  • 1 - 9:24 Nodejs Refresher Scott Moss begins the course with a brief refresher on Nodejs. He talks about the various ways it can be used and explains Node’s package manager, NPM.
    - https://github.com/FrontendMasters/api-design-node
  • 9:25 - 19:43 CommonJS Node uses CommonJS as it’s module loader. Scott explains how to use the require() method to load built-in modules, Node modules, and user-created modules. He also talks about the differences between using exports and module.exports when creating a module.
  • 19:44 - 22:37 Executing Node JavaScript can be executed in Node directly from the command line or by passing the name of a JavaScript file as a command line argument. In either case, the JavaScript code will behave similarly to being run in a browser. Scott demonstrates both methods of execution.
  • 22:38 - 30:44 Express While Node has a built-in server module, using it to create a basic server can be overwhelming. Instead, Scott will be using Express to handle server requests. Express is one of many server frameworks for Node which greatly simplifies the process of creating your own API.
  • 30:45 - 38:28 Using Express Scott walks through a simple Express example. This example is a basic HTTP server listening on port 3000. The server has a couple routes for handling both GET and POST requests for a /todos API.
  • 38:29 - 44:51 Exercise 1 In this exercise, you will create a basic web server with Express. The server will send back the index.html file when it receives a ‘/‘ request. It will also send back JSON data when a /data request is received.
  • 44:52 - 55:39 Exercise 1 Solution Scott walks through the solution to exercise 1. He also shares a few workflow tips like using nodemon to auto-restart the server after each change.

• REST APIs

  • 55:40 - 59:54 Getting RESTful An API that’s considered to be RESTful should be stateless, use HTTP verbs, and expose a directory-like URL pattern for routes. While REST is not a standard, it’s how most of the modern web APIs are built.
  • 59:55 - 1:10:11 Anatomy of a REST API Scott discusses how to create a REST API. He begins with a model of the data. Then he makes the routes to create, read, update and delete (CRUD) the data. Once these resources are in place, the API can be created.
  • 1:10:12 - 1:16:54 Exercise 2 In this exercise, you will create all the necessary routes for the ‘lions’ API. Before beginning the exercise, Scott explains the middleware used to serve the client application.
  • 1:16:55 - 1:27:20 Exercise 2 Solution Scott walks through the solution to exercise 2. He begins with the GET and POST routes which handle the retrieval and adding of lions.
  • 1:27:21 - 1:35:47 Exercise 2 Solution, continued Scott finishes his explanation of the second exercise by covering the PUT and DELETE routes which update and remove lions.

• Middleware

  • 1:35:48 - 1:45:15 What is Middleware Middleware is a function that has access to the request object, response object, and the next function which points to the next middleware. Since the request and response object available, middleware is able to change these objects, run any code, or even end the request/response cycle.
  • 1:45:16 - 1:58:31 Types of Middleware In Express 4.x, there are five different types of middleware: 3rd Party, Router, Application, Error-handling, and Built-in. Scott explains a few of these types and shares a few code examples of their use. He also spends some time talking in-depth about error handling.
  • 1:58:32 - 2:05:08 Middleware Examples There are many ways mIddleware can be used within an application. Scott shares a code example that demonstrates a few of these ways. He also compares 3rd party middleware with middleware created within the application.
  • 2:05:09 - 2:11:36 Exercise 3 In this exercise, you will add a number of middleware methods to the server. The middleware will handle incoming lion requests, errors, and additional routing.
  • 2:11:37 - 2:23:00 Exercise 3 Solution Scott goes through the solution to exercise 3.
  • 2:23:01 - 2:30:05 Routers Express 4.x allows for more than one router in an application. This means developers can use a router in a more modular way. A router can have it’s own routing, stack, and functionality. Scott demonstrates this by showing a code snippet with an encapsulated todosRouter.
  • 2:30:06 - 2:32:22 Exercise 4 In this exercise, you will make a new Router for the tigers module. You will then mount that router in the application alongside the lions module.
  • 2:32:23 - 2:39:20 Exercise 4 Solution Scott walks through the solution to exercise 4.
  • 2:39:21 - 2:46:05 Error Handling Before moving on, Scott makes a few code optimizations to demonstrate who to simplify the routing syntax. He also talks briefly about error handling. Node doesn’t output any error details by default so Scott adds an error handler and explains how to use the Node debugger/inspector.
  • 2:46:06 - 2:51:39 Audience Q&A Scott spends some time answering audience questions. These questions range from other debugging tools, routing tips, and logging.
  • 2:51:40 - 3:02:18 Audience Q&A continued Scott continues his audience Q&A with a demonstration on how to use ES2015 features with Node.

• Testing

  • 3:02:19 - 3:14:39 Testing in Node Unit testing in Node is very similar to unit testing in the browser except for the absence of the DOM. To make the testing process easier, Scott recommends exporting the main application so it can be used by the testing framework. He also covers a couple strategies for writing API tests and saving development dependencies.
  • 3:14:40 - 3:20:39 Node Environment Variables Scott spends a couple minutes talking about Node environment variables. All environment variables are stored on a global process.env variable. One important variable is the process.env.NODE_ENV which determines if an application is running in a production or test environment.
  • 3:20:40 - 3:32:25 Exercise 5 In this exercise, you will write tests for all the CRUD routes in the application. You will use Mocha to run the tests and view the results.
  • 3:32:26 - 3:43:56 Exercise 5 Solution Scott demonstrates his solution to exercise 5. He also shares a few variations of the tests since there are many ways they could be written.
  • 3:43:57 - 3:55:33 Testing Q&A Before moving on, Scott answers a number of audience questions on testing strategies and demonstrates a few alternative approaches.

• Organization & Configuration

  • 3:55:34 - 4:01:14 Application Organization Scott talks about the component approach he uses when organizing application files. Within a component, models describe how the component’s resources will look, controllers access these resources, and routes help define how the API will be exposed.
  • 4:01:15 - 4:13:17 Configuration One important role of the configuration file in an application is to determine the environment in which the application is running. Scott explains how using the process.env.NODE_ENV allows him to set specific environment URLs, application secrets, or logging settings.
  • 4:13:18 - 4:29:36 Exercise 6 In this exercise, you will be loading the envConfig variable with the correct configuration data based on the environment. You will also be properly configuring the routers. Before beginning the exercise, Scott briefly walks through the application architecture.
  • 4:29:37 - 4:44:47 Exercise 6 Solution Scott walks through the solution to exercise 6.

• Mongo

  • 4:44:48 - 4:55:27 Mongo Introduction Mongo is a NoSQL document store. Data does not have to be modeled in advance with Mongo which makes it very easy to setup and use. Scott demonstrates how to create a new collection and insert records using the Mongo command line interface.
  • 4:55:28 - 5:00:19 Using the Mongo with Node Mongoose is a JavaScript diver for Mongo that will expose an easier-to-use API inside a Node environment. Mongoose will also add support for APIs like Promises and allow the data to establish relationships and schemas.
  • 5:00:20 - 5:02:55 Exercise 7 In this exercise, you will connect to a Mongo database. Once the database connection code is in place, you can verify the connection through the router.
  • 5:02:56 - 5:06:36 Exercise 7 Solution Scott walks through the solution to exercise 7. He also briefly discusses collection naming conventions in Mongoose.

• Data Modeling

  • 5:06:37 - 5:12:56 Schemas Mongoose schemas can be used to add structure and validation through a process called data modeling. Scott walks through the code to create a schema and demonstrates the different ways properties can be defined.
  • 5:12:57 - 5:19:06 Schema Types Scott walks through a more complex schema definition to introduce all the different data types that can be used when defining properties. These data types include Boolean, String, Number, Date, and Buffer. Scott also talks about nested objects and additional validation rules like min and max.
  • 5:19:07 - 5:30:22 ObjectId The ObjectId type is used to create primary/foreign key relationships between objects in the database. While this property looks like a String, it’s actually an Object with a unique reference. Scott explains how to use the ObjectId type and answers a number of audience questions about it.
  • 5:30:23 - 5:38:55 Blog Schema Representation Scott spends a few minutes looking at a JSON representation of the schema for the blog application. After walking through the properties, he answer a few questions about migrating data to a new schema.
  • 5:38:56 - 5:40:02 Exercise 8 In this exercise, you will design schemas for the category, post, and user models in the application.
  • 05:42:42 - 05:56:34 Exercise 8 Solution Scott walks through the solution to exercise 8.

• Querying

  • 5:40:03 - 5:51:41 Querying Data with Mongoose Mongo has a sophisticated querying language with a lot of options. The find() method, for example, is passed a query object. While there are variations of the find() method, most method calls look similar. A callback is passed as a second parameter. Scott demonstrates how to use the callback to step through, update, and delete documents.
  • 5:51:42 - 5:56:10 Populations Since Mongo is a NoSQL database, there are no join tables. Relational data instead uses Populations. Populations behave like a join table, but are used at call-time.
  • 5:56:11 - 5:58:15 Exercise 9 In this exercise, you will build out the controller files for each module.
  • 5:58:16 - 6:09:10 Exercise 9 Solution Scott begins the walk through for exercise 9. First he starts with the params method in the categoryController.
  • 6:09:11 - 6:14:57 Exercise 9 Solution, continued Scott continues the exercise 9 solution by explaining the get method in the postController and the getOne method in the userController.
  • 6:14:58 - 6:25:08 Creating Promises Before moving on, Scott spends a few minutes explaining the basics of promises. He starts with a simple example showing how to create a promise with the native JavaScript API.
  • 6:25:09 - 6:31:22 Consuming Promises Using Node’s file system API, Scott demonstrates how to consume and Node-style callback as a Promise. He wraps the API call in a native Promise and returns a call to the resolve or reject methods.
  • 6:31:23 - 6:43:18 Nested Promises As promises get more and more complex, additional methods can be used to make their consumption cleaner and more readable. Scott explains how to error handle individual promise calls. He also introduces the all() method which will consume an array of promises.

• Authentication

  • 6:43:19 - 6:50:09 JSON Web Tokens This application will use JSON Web Tokes (JWT) to handle authentication. JWT will replace the traditional cookie/session authentication used in traditional web applications. Scott introduces JWT and talks about how they will be used.
  • 6:50:10 - 6:58:39 Using JWT Scott shares a brief code snippet demonstrating how to work with JWT. A token is generated by hashing some identifying user data with a secret key. During subsequent requests the server will use the same secret to verify the request’s token is valid.
  • 6:58:40 - 7:05:51 Usernames & Passwords Scott begins explaining the process of creating a new user. Users will supply their username and a password. A hashed version of the password will be stored in the database. When a users logs on in the future, the submitted password will be hashed and compared to the stored password in the database.
  • 7:05:52 - 7:15:34 Authentication with Middleware Mongoose allows new methods to be added to the Models and Documents in the application. Scott explains how these new methods will allow a middleware API to be built around the authentication process by using lifecycle events like “before save” and “before validation”.
  • 7:15:35 - 7:16:57 Exercise 10 In this exercise, you will connect the routes with their controller methods.
  • 7:16:58 - 7:25:40 Exercise 10 Solution Scott walks through the solution to exercise 10. He also demonstrates how to create utility functions for generating the routes.
  • 7:25:41 - 7:32:42 Executing CRUD Operations Before moving on to authentication, Scott spends a few minutes testing the current codebase by executing a few CRUD operations from the command line. He adds a new user, creates a category, and creates a blog post. He also explains how populations work.
  • 7:32:43 - 7:41:29 Authentication Configuration Before moving into the next exercise, Scott gives a quick overview of the configuration changes he made to integrate the authentication module. He explains the need for a couple utility functions as well.
  • 7:41:30 - 7:53:59 Exercise 11 In this exercise, you will implement the getFreshUser and verifyUser methods in the “auth” module. You will also implement a sign-in method that will generate a token to be sent back to the client.
  • 7:54:00 - 8:07:04 Exercise 11 Solution Scott explains the solution to exercise 11.
  • 8:07:05 - 8:16:38 Testing the Authentication With the authentication now fully implemented, Scott spends a few minutes testing the API with one of the seeded users in the database. He also answers a few questions about writing a custom authentication layer versus using other third-party libraries.

• Securing Routes

  • 8:16:39 - 8:24:26 Identifying Sensitive Routes Not all routes in an API should be publicly available. Scott goes through the post, category, and user modules to identify routes which require authentication. He also explains edge cases where a route should only be available to a single user versus all users.
  • 8:24:27 - 8:35:54 Understanding CORS Scott installs the blog user interface which will be used throughout the rest of the course for working with the API. Since the blog interface runs on a separate port, CORS will need to be enabled to allow access to resources form different origins.
    - https://github.com/frontendmasters/angular-components
  • 8:35:55 - 8:41:57 Testing the UI Now that CORS has been added, Scott spends a few minutes testing the new user interface. He demonstrates the admin and sign-up functionality and explains what routes still need to be secured.
  • 8:41:58 - 8:50:19 Exercise 12 In this exercise, you will secure the routes that create a new blog post. This way, only logged in users will be able to create posts. You will also need to enable CORS so the UI application can communicate with the API.
  • 8:50:20 - 9:05:38 Exercise 12 Solution After answering a few audience questions, Scott walks through the solution to exercise 12.
  • 9:05:39 - 9:12:49 Exercise 12 Solution, continued Scott continues talking about the solution to exercise 12. He demonstrates the working user interface and uses the Chrome Developer Tools to show the authorization tokens in action.

• Deployment

  • 9:12:50 - 9:21:37 Deployment Overview While deployment of a Node application is relatively easy, there are some things to consider. Scott runs through some of his recommendations like using ENVs for secrets and ensuring the platform has access to all necessary build tools.
  • 9:21:38 - 9:33:54 Deploying to Heroku Scott will be deploying the blog application to Heroku. He logs into his Heroku account and creates a new application. Scott then demonstrates the process of pushing the application to the deployment server and debugging any issues.
  • 9:33:55 - 9:44:00 Configuring the Deployment The Heroku deployment server is missing some of the necessary environment variables for the application. Scott demonstrates how to configure the server for the correct environment and database URI.

• Q&A

  • 9:44:01 - 9:56:14 Q&A Part 1 Scott begins the final Q&A by answering questions about when to use query string variables, how to identify quality node libraries, and the scalability of Mongo.
  • 9:56:15 - 10:05:44 Q&A Part 2 Scott continues the final Q&A with some ways to scale the MEAN stack, recommended resources for beginners, and NPM modules for interfacing with other databases.
  • 10:05:45 - 10:18:37 Q&A Part 3 Scott wraps up the course with a few last Q&A questions about service-oriented architectures, using different third-party services, recommendations for style guides or code-quality checkers, and profiling/debugging Node.