Lesson Description

The "Protocols" Lesson is part of the full, Backend System Design course featured in this preview video. Here's what you'd learn in this lesson:

Jem discusses HTTP protocols, its stateless nature, and the role of context and metadata. He also covers WebSockets for bi-directional communication and compares protocols like gRPC, REST, and GraphQL.

Preview

Transcript from the "Protocols" Lesson

[00:00:00]
>> Jem Young: protocols HTTP, the protocol of the internet. Know it, love it. HTTP has headers, you can cache it. Things to know, the web is stateless. Yeah, we take that for granted as an assumption, but when I first got into programming, I was doing C and I switched over to web programming. I was like, this is so difficult. I can't I just remember the thing I was just doing? You can't, every transaction is on the internet, is brand new.

[00:00:30]
It's amazing, until you give it the context and metadata. Really important to know about programming. Not all protocols are stateless, HTTP is. But HTTP is awesome because it's simple, it's human readable, it's supported by all the browsers. The law of the internet. WebSockets. Anybody ever use the web sockets? I run a service. Yeah. WebSockets are. What? Bidirectional communication goes back and forth.

[00:01:05]
Reuses the connection. Yeah, and it's also auto frames. It's persistent. Um, it's generally low latency because that connection stays open, so it doesn't have to do the handshake every single time, but a WebSocket is stateful. It can remember what happened previously. Uh, why don't we use a WebSocket? Chat apps. Yeah, I use it for like status updates, just to refresh UI. Interesting. I have a proposal for you.

[00:01:37]
Maybe you want a Server-Sent Events. I actually do, but when I built it, that was not well supported. Totally OK. My bias, my default is Server-Sent Events or WebSockets, it's just like, I don't know, it's cool, it's easy. If you need it later, you can do bidirectional communication. I have a goal to replace it with that because 90% of my use case is that, but I don't have time. Yeah, I was actually going to share like a quick anecdote about something I had to do for work a couple of months ago.

[00:02:09]
Basically we were running into a problem where we had too many servers and events and like because of how the protocol defines, if you have too many of them, it makes the entire browser lock up. So I had to make a custom one-way WebSocket implementation, a one-way WebSocket. Yeah, I don't. I've never used Server-Sent Events personally. I think WebSockets are pretty fairly, very easy to implement now.

[00:02:35]
But I can see there is overhead to a WebSocket because you are maintaining that connection and sometimes it's not what you want. So a good example, like a news feed, you're just pushing, pushing things. You don't care what the client does, if he receives it or not, it's not going to say anything back to you, it's just, you know, publishing events. The best implementation I've seen of this so far is LaunchDarkly.

[00:03:01]
If you look behind the scenes, how they do their stuff, they're heavily using this. They probably have other ways they do it too, but I've seen that a lot. That's pretty cool. Yeah, I don't know many people who know much about Server-Sent Events, honestly, but it's a thing. We might want to use in some cases. I wanted to use it in 2016. It was not well supported. Yeah. You know, they're somewhere between a WebSocket and an HTTP, but unless you're on the bleeding edge here, you probably can use WebSockets and no one's going to ding you for it, really.

[00:03:39]
Well, the thing is Server-Sent Events are actually older than WebSockets. Today I learned. I didn't know that. Why isn't anybody using them, just like web components? You know, any day now they'll be popular. My coworker was recently looking into this and he said when you Google it, it's actually just really hard to find good information on it because the words are so common or something like that, like he had to dig a little bit.

[00:04:08]
I never thought of that. I once interviewed for a company called Live and the first thing I said was you should change your name because it's literally impossible to find you on the internet. They're like, yeah, you know, we've been thinking about it. Is that a? Oh, no, Live. Yeah, that was the company name Live. Yeah. And gRPC, gRPC, it's great. It's fast. It's a contract between the two. Browsers don't support it natively.

[00:04:40]
You need a library to interpret that. It only works over HTTP/2 and it uses protocol buffers which are not readable by the average mortal. I don't think everybody can read protocol buffers, but they are very fast, very fast and good for low latency environments. So a lot of our IoTs, it's just making a remote procedure call which is just like, hey, I'm doing something in action. Oh, I forgot the grandfather of them all because SOAP technically predates REST, but REST, REST is old faithful, very.

[00:05:14]
I won't say straightforward because I don't, that's not a good term because maybe it's difficult. For me, REST is pretty easy to reason about. There's a set of actions either getting, you're putting, you're posting, you're deleting. What does that sound like, a CRUD app, doing the same thing? Yeah. That's what kind of the origination of it. It's supported by all browsers by default, baked into the inspector.

[00:05:45]
Yeah, easily cached. It's pretty straightforward, I'll say. I shouldn't say easy though. And we have crafted you well. And yes, I know we're comparing different things. We're comparing APIs to protocols here, but I'm throwing it all in their protocols. GraphQL, lots of people use GraphQL today. Maybe when they should, they should probably use REST at times, but we use GraphQL because it has great developer ergonomics.

[00:06:13]
The API is self-documenting, you get nice autocomplete with GraphQL. However, there is overhead to it. It's not natively supported by browsers, so you gotta interpret all that. You are setting things as a JSON blob which limits the kind of data you can send in GraphQL. You can get around that. But really it does make your data sources more complex to reason about and that's what you want GraphQL for.

[00:06:36]
If you got a lot of data sources, example here is like a Facebook homepage. News feed, pictures, ads, all that in one go. GraphQL is great for that because it can fetch all that and make it look like one endpoint versus REST, we'd have to make a bunch of calls. But generally your options are coming down to REST and GraphQL, you know, but this is my bias as a frontend developer who hasn't done a lot of gRPC or Server-Sent Events.

[00:07:09]
Here's a quick cheat sheet. Pretty, I won't say straightforward. I gotta stop saying straightforward. Is it service to service? Generally, when you hear that, I think gRPC, that's what we use on my team, but only in between services, not front-end to, not our back end to front-end. Does it need to be real-time? I know, WebSockets, right? Yes, we just talked about this, but you can also use Server-Sent Events.

[00:07:33]
We should all make it a mandate of ourselves to push ourselves and use a Server-Sent Events for something. And does the data, is the data source complex or is it pretty straightforward? Or another way of reasoning about that is, does everybody need to know everything at the same time? Because that's what REST gives you. Every request you make, unless you're adding query parameters and special addendums to the API request, everybody's getting everything.

[00:08:00]
Sometimes you want that, there's many cases you don't. So you're choosing between GraphQL and REST generally. And here's a big general advice, someone's going to be like, well, actually, Joe, and yes, you can use a lot of these for a lot of things, but these are just, hey, I'm going to interview real quick, which protocol am I going to use? Got it. Easy. If you're wrong, you're wrong, it's OK.

Learn Straight from the Experts Who Shape the Modern Web

  • 250+
    In-depth Courses
  • Industry Leading Experts
  • 24
    Learning Paths
  • Live Interactive Workshops
Get Unlimited Access Now