James Halliday
Substack
Networking and Streams
3 hours, 7 minutes CC
Course Description
You’ll be introduced to the fundamentals of the built-in stream Node.js module, and learn how to code streams using backpressure to combine multiple data sources and sinks for better data processing. Learn how to use curl and netcat, different stream types - readable, writable, transform, duplex - plus write code for TCP, HTTP, and Websocket servers and clients using Node.js. Finally, learn how to build multiplex streams! This course is ideal for moderately experienced engineers who want to learn how to leverage networking fundamentals and protocols to master the art of building streams in Node.js.
This course and others like it are available as part of our Frontend Masters video subscription.
Preview
CloseCourse Details
Published: September 13, 2017
Topics
Learn Straight from the Experts Who Shape the Modern Web
Your Path to Senior Developer and Beyond
- 200+ In-depth courses
- 18 Learning Paths
- Industry Leading Experts
- Live Interactive Workshops
Table of Contents
Networking, Servers, and Clients
Section Duration: 41 minutes
James Halliday introduces Bash by placing its context with the history and background of Unix, an operating system first developed at AT&T Bell Labs in the 1960s.
James introduces the concept of protocols, which are languages that allow computer programs to speak to each other. Examples of protocols include HTTP, SMTP, IMP, and SSH.
James talks about a server, which is a computer listening to incoming connections, even if they are coming from local connections. Clients are computers that are initiating connections with a server on the other end.
Netcat (often abbreviated to nc) is a computer networking utility for reading from and writing to network connections using TCP or UDP.
By showing a simple page request to Google from the command line, James introduces the Hypertext Transfer Protocol (HTTP), which is how web servers and browsers communicate with each other.
James demonstrates how HTTP Post is used to send information through forms.
James shows how to use the curl command, a tool to transfer data from or to a server with the results from a curl request of presented within the terminal. James takes questions from students.
James demonstrates how to set HTTP headers within a curl request.
By sending an email through localhost, James introduces SMTP, the protocol used to deliver email messages.
James introduces Internet Relay Chat (IRC), a text-based chat protocol that is still very popular among programmers.
James takes questions from a student about setting a nickname in an IRC.
After mentioning that the previous protocols have been text-based, James reviews binary protocols like SSH. Then James shows how to inspect protocols to review data traveling coming out and into your machine.
Streams
Section Duration: 36 minutes
James introduces streams as a method to move data around networks or even on the same computer.
To transform a data with a stream, James shows how to convert text case within a file and display the result in standard output. James takes questions from students.
After installing through2 that allows the creation of streams quickly.
James takes questions from student about streaming
James reviews the differences between through2 and native npm for using streams.
James introduces concat-stream, which is used to collect all the data from a stream into a single buffer.
James takes questions from students about starting and closing streams. James also demonstrates how to display a message that a buffer is too big instead of closing out the program without warning.
Stream Types
Section Duration: 32 minutes
James reviews the different types of streams: readable, writable, transform, and duplex.
After examining the methods for creating writable streams, James demonstrates how to build one.
James looks at readable stream methods, which are used when supplying data. Readable streams can also be paused and resumed.
James reviews the transform stream, which is a particular kind of duplex stream used to operate on a stream in transit. Then James looks at duplex streams, which can accept and deliver content at the same time.
After taking questions from students about different types streams, James provides an example of a duplex stream. Then James takes questions from students.
Based on a question from a student, James builds a simple VPN using cryptographic core streams methods.
James introduces object streams with objectMode, which allows use of any kind of object except null into a stream.
Core Streams
Section Duration: 22 minutes
James reviews the core streams that are available in node.
After reviewing HTTP Core Streams, James builds an server and client and have them talk to each other.
James discusses other cryptographic core methods and demonstrates how to incorporate a hash into a stream that can be used to securely identify unique information.
To use Gzip and Deflate/Inflate, James reviews the Zlib core methods.
James introduces split2, which breaks up a stream and reassemble that stream so that each line is a chunk.
Web Socket
Section Duration: 10 minutes
James illustrates how to put streams in the browser with HTML5 websockets using node Streams API.
After setting up the websockets to work in the browser, James shows how to streams in node with a node websocket client.
Stream Modules
Section Duration: 14 minutes
James reviews modules found to be useful in development: collect-stream for putting streams into an array and performing unit tests; from2 for ready stream with a pull function; and to2 for a writable stream with a write and flush function;
James introduces a duplexify, which creates a writeable and readable stream into a single streams2 duplex stream.
James discusses error propagation and how it leads to server crashes. To contribute to creating stronger streams, James uses the pump module that helps clean up streams to handle errors gracefully. The pumpify module does similar error handling as pump, but also provides a readable and writeable stream. James introduces end-of-stream, which is a node module that calls a callback when a readable/writable/duplex stream has completed or failed.
Remote Procedure Call and Multiplex
Section Duration: 12 minutes
James introduces Remote Procedure Call (RPC), which is a protocol that one program can use to request a service from a program located on another computer on a network without having to understand the network's details. Then James shows how to set up a simple RPC system.
Due to limitations of the number of streams a browser can have at one time, James reviews how to use multiplex to send many streams over a single stream.
Wrapping Up
Section Duration: 17 minutes
James takes questions from the audience about using WebRTC with webrtc-swarm.
James takes questions from the audience about tag template literal, the HTML parser, and application frameworks.
James talks about the future of JavaScript and WebAssembly.
Learn Straight from the Experts Who Shape the Modern Web
- In-depth Courses
- Industry Leading Experts
- Learning Paths
- Live Interactive Workshops