Identify Requirements

[00:00:00]
>> Jem Young: How to build anything. A bit bombastic of a title of its own, maybe some ego in there. How do we build anything? These are elements or strategies and steps that will help you design any sort of system. Maybe not a car, actually a car. I gotta stop using that example. Let me find another example. Someone give me another example that's not a car, anything at all. A rocket. Thank you, Mark, even better.

[00:00:31]
The stakes are much higher there. We can take a set of common components and almost anything breaks down into high level components when you think about it. We talked about my bathroom remodel, a lot of different components, but at the end of the day they break down to, you know, handles for the faucets, the sinks, the shower, the drains. Maybe a bathtub and a lighthouse, a lighthouse. You said how to build anything.

[00:01:04]
I know, but now I'm thinking like what are the components of a lighthouse? That's really interesting. Anyways, we'll come back to that one. But we learned everything has these just common parts to it. And if you know how to speak the vocabulary of these common parts, you can describe anything of any complication, even lighthouses or rockets. What are the core elements of good system design? The first one is translating the business requirements.

[00:01:36]
I'll say it again because this is really crucial. Translating the business requirement is the core element here. No, Jim, it's the diagrams and the boxes and I've got 30 databases of different types and CDNs and caches around the world. And it's awesome. It's amazing. The diagram takes the entire whiteboard. Not the important part. You'll get there. The important part is translating business requirements.

[00:02:06]
If you can do this effectively, you will excel in any role you want. And I'm not even in tech in anything. If you understand what someone is asking you to do, what are the details of that, you will be effective in, you know, anything you're going to do. The difference between, in my mind, one of the differences between, say, a senior engineer and a staff engineer is a senior engineer is going to take a problem and be able to break it down.

[00:02:33]
So you're like, oh, I've seen this before. Here's how you design it, you know, it's fault tolerant, it's got tests, it's got good standards. But a staff engineer is going to say, what are we actually trying to solve here? What is the relationship between me or my team or my org to the business? And how do I apply those relationships to the thing I need to solve? And that's the difference I've seen in real life as a manager.

[00:02:58]
An effective staff engineer is always, what are we trying to solve? Versus our instinct is like, let me jump in and code. That's why I sink too a lot of times. I'll figure it out when I look at it, I can see the code, but the ability to not look at code and still understand the problem and be able to reason that out in your head, that is a skill, and we want to build some of that today. After that is designing the API and the architecture.

[00:03:22]
So, more on the diagram side. Now we're getting into, OK, we know the components, how do they all connect to each other? And then after that, we want to get more into the edge cases handling. Making trade-offs, handling failure, understanding the actual technology, when it matters, when it doesn't matter. When does the difference between a MySQL database and a Postgres database matter? Sometimes it does, sometimes it doesn't.

[00:03:48]
Depends on what you want to do. But that's always the last step. Once we have everything laid out, it's a lot easier. But if you're in the middle of, say, step two, designing the architecture and the diagram, you will get stuck. You will get stuck if you didn't build up to it. And I've said it before, I failed design interviews. I know, I'm young, ah, gasp, my hero. Yeah. And you know why I failed them?

[00:04:13]
Because I'm like, the diagrams what matters. That's the actual output that they're looking for it. I kept, I jumped into it, and then I got to a certain step and I realized, oh, I didn't think of that. And I should have just slowed down. So, let's slow down a little bit and let's focus on translating these business requirements. It doesn't seem fun at first, but good system design is really, really boring.

[00:04:43]
You'll see. So we're going to design a to do app. Why a to do app? What does every intro to programming tutorial have, what is it always? It's a, thank you to do app. It's one of the simplest types of applications you can think of, not highly complex. The data is pretty easy to reason about. The read-writes are sort of balanced. So let's design one because it can't be that hard, right? Sorry, that's my evil grin.

[00:05:20]
All right. A to do app, what are the requirements? High level, keep it high level. We'll say, you know, there's only 4 or 5. Say 4, complete to do. You have to be able to view a list of to dos. I heard complete news, view the lists. Create to do. Create to do, yeah, mark them off, mark them off, yeah. Would you say deleting entity would be in scope? Yeah, you gotta be able to do that, otherwise, what's the point?

[00:05:56]
Well, sorry, I meant specifically like if you accidentally make a to do with a bunch of typos or something and you're just like, OK, I don't want this, let me get rid of it. Yeah, delete it or edit it, yeah. Is that it? Really maybe. Yes, someone said it, create, read, update, delete. I said to do task. So we need, what are the core things we need to do? We need to be able to create a to do, we need to read a to do, we need to edit it, and we need to delete it.

[00:06:35]
That's it. These are the core requirements, when you think about it. Pretty straightforward, right? Yeah, yeah, totally. So how would you design this? I won't ask you to go to the whiteboard, but how would you design this? I think probably the first thing I would ask is like, are we writing a general to do app that's like something like a simple notepad? Is it specialized for like a specific domain?

[00:07:01]
I like where your head's at, Michael. We're not quite there yet. But generally, how would you design something like this? Like what are the components you'd use? Might start simple and just build a monolithic app, monolithic app, yeah, client-server database, pretty straightforward, yeah. OK, this wasn't fair. I sort of tricked you. I promise I won't trick you anymore. I could build a to do app just like this.

[00:07:27]
I don't need anything because we didn't specify how many users does it need to be accessed from different devices. We just said build a to do app. So I can do that all in the browser, local storage, IndexDB if I want to feel ambitious. Nothing to it. I won't say nothing to it, but not very complicated of a system. So that wasn't fair. But I was illustrating a point on to what Michael was saying, we have to ask questions.

[00:07:53]
We have to say, what are the requirements here? Yeah, well, I was also going to ask, or I was going to joke, I feel like the staff engineer answer might just be to say, hey, like, do we actually need it to do that, or can you just use like a notepad? Yeah. Yeah, good staff engineer is like, why do we need to, are you sure we need to build this? And they get out of as much work as possible. That's how you get ahead.

[00:08:23]
Yeah, if we're building just a new app for myself, I don't need anything complicated. I don't need a server or database. Do it all in the browser. So we're building a to do app, we say, cool, I know I need to create a to do, edit a to do, read a to do and delete a to do. Straightforward, right? Let me dive right in. Now, what are the core features that we actually need to support with a to do app?

[00:08:49]
And we said them, but we didn't get any detail around them, and the details matter. So when we're handed a prompt, design a studio app, design anything. The first thing we have to do is, what's the scope of the problem. We didn't establish how many users there are. That's an important scoping question. There's a difference between a one person to do app and a billion person to do app, very, very different systems.

[00:09:14]
After that, we want to design the high level architecture. Just lay out the core components, the ones we're already familiar with, nothing fancy, just to make sure you're on the right track. And then we go into the key challenges and trade-offs once we have those core architectures or the details. And this is a good strategy. This is a strategy that you could apply and solve a problem in an hour, which is about what you have, usually less than an hour to solve a system design.

[00:09:45]
Scoping 10 to 15 minutes, high level architecture, say 20-30 minutes. Then the trade-offs discussion, you know, variable, say 15 minutes at the end. That's the strategy we're going to apply. But the scoping is the most important part. If you don't scope it, you might as well just throw away the interview. You can just waste your time. You're like, cool, I built this thing. They're like, yeah, but you didn't.

[00:10:14]
You didn't ask if there is a mobile client. Is this a mobile app? Changes the dynamics of what you're building. So that's a fast chapter, but I want to emphasize, we have to ask questions. We always ask questions. Don't make assumptions, do not make assumptions. Assumptions will sink you. Yes, so scoping that's that encompasses determining functional and non-functional requirements. That's the next right there, yeah.

[00:10:49]
So the first thing we have to do, let's understand the problem. We like to call these, these are called functional requirements, maybe a different way of thinking of this, maybe feature requirements. What is the function of the application? What are we trying to do? What is the core thing we're trying to get to? Again, we have to ask, what are the requirements we're trying to build for? What are the features needed for it to work?

[00:11:13]
What is important and what's not important? We talked about, oh man, I'm doing it again, car example, got to get another one. But what's important there in these components, the common components, what is the important thing? Again, with scoping it down to an hour, you don't have time to get into detail about everything. So what is the thing they're trying to get you to solve? And you have to ask these questions to get there.

[00:11:40]
A system design interview, it really is kind of two different things. One, it's, you know, showing off your knowledge. What do you, where's your strength? Your strengths are databases, your strengths, big picture thinking, you're more focused on front-end delivery. It's a chance to showcase some of that, but it's really also about working through ambiguity. It's an ambiguous problem. They're not going to give you a lot.

[00:12:07]
What I've gotten? A real question, design a ticketing system. That was it. That was the only prompt. Like what, what kind of ticketing system? What are we like movies, concerts, fancy restaurants? But that's all you're going to get in these interviews. It's just a very simple prompt. The only way you can get from there to a diagram is if you ask questions. And also important steps here, listen to what they're saying.

[00:12:34]
They're not going to tell you something that's not material. Listen to what they're saying. Write it down. Always write it down. So you say, here's what I heard, this, this, this and this. And after you write it down, you say you can step back and say, OK. Yeah, now I have a good understanding of the problem as written. From there you have the space to explore, and there you have the chance to showcase your knowledge, but it's only when you understand that problem.

[00:13:01]
I know I'm hammering this over and over again, but again, it's the common mistake. I want to get to diagramming. Just don't. We're not going to get there yet. So when it comes to functional requirements, there's two different kinds. There's functional, the features, describes what a system can do. How does the user interact with the system, the user experience, the business requirements of it, functional features, what the system does.

[00:13:33]
The non-functional is the tricky part, sometimes, depending on your perspective. Non-functional describes how the system should perform. What is the quality we need to maintain? Is it important that the system is fast, or is it more important that the system is consistent? Can you have all of these? Not really. You can, you can get to most of it, but not all of it. That's why you have to understand a non-functional because you're here you're saying, what are the trade-offs we're willing to make with this system.

[00:14:03]
If I'm building a video streamer, what's important there? Is it that when I upload videos, they're available to users really quickly, that's the important thing, the speed, or is it we want to maximize, say the quality of the video for the user at the end. So in that case, we need to encode a lot of different types of videos in order to give that to people. So again, you, the non-functional I find are more of the more challenging ones, because you have to know the right questions to ask to get there, because those are the ones constraining your system ultimately.

[00:14:42]
So basic functional requirements. Let's ask, what are the core features that we're trying to develop? What should the user be able to do is an easy one, shortcut to getting to functional requirements. Who are the users of this app? Is it people that are very skilled, they're on fast machines, or is it, hey, we're targeting people with cell phones and in high latency environments. Are there different kinds of users?

[00:15:12]
Do they all have the same level of permission? Are there users and administrators and then owners? Something like that? Good question to ask up front. How do the users interact with the system? We talked earlier about smart refrigerators. That's an interesting interaction. But are, you know, what type of device are they using? Are they in a car? On a phone? Is it a VR headset? Good question to ask.

[00:15:39]
What are the devices? If you get that, especially devices, you can scope the problem down really quickly. Instead of, oh, I need to support mobile and then a browser and a TV, different constraints there. But if you can get that down, makes your job a lot easier. And what are the edge cases in the functionality we need to consider? Do we need to say, I don't know, sign a banking app, log people out in a certain time frame.

[00:16:07]
That's a feature. What are the constraints that we're operating under? Example of functional requirements for our awesome pizza shop. We'd say something like it's usually an action verb of some sort. The user should be able to, the user must be able to. The user should be able to view a menu of pizzas and toppings. I don't think any of us got that in our initial design of pizza shop. What's the, how's the user going to order anything?

[00:16:33]
We can't assume that they know the menu from the back of their, like, you know, back of their head from heart. The user should be able to order pizza. Good to write it down. Yes, we're designing a pizza shop. They should be able to order a pizza. Or maybe it's a pizza delivery and the orders are preprogrammed, sent out by drones or something. I don't know. You don't know until you write these things down, and the other person's going to be like, yep, that's important, that's not important.

[00:17:06]
What are the constraints? Can the user order multiple pizzas? How many pizzas can they order? 100? 10, 1? That matters. You know, if the user can order 100 pizzas, you have to think very differently about the number of ovens you have. How many sizes of pizza are there? Is it a million, infinite? Does it matter? Is there one? How many toppings did a pizza have? Yes, this is a contrived example of a pizza shop, but these are examples of functional requirements.

[00:17:32]
We don't assume anything. That's the thing, we don't assume. We're working through an ambiguous problem. So we have to ask the questions to get this level. And again, you write it down because what's the worst thing they're going to say? That's wrong. Cool. Now I understand the scope of the problem a little bit better. And it's kind of an art to asking the right questions, but there's basic ones you can ask, you know, users, constraints, those sort of things.

[00:18:00]
Yes, I was going to ask, like, do you have any recommendations for how to document some of the functional requirements you identify? So it's like, OK, for example, like we say, OK, we only support 3 slices of pizza right now. That's fundamentally baked into the system, no pun intended, but like, let's say like a year or two down the line, somebody goes, Oh, why do we have this hard requirement? How do we, like, why did we design it this specific way?

[00:18:30]
Like, is that something you try to do within the code itself in like documentation pages or like what would you recommend? Yeah, I love what you're thinking, Michael. You say a good example would be, hey, there's only 3 slices of pizza. Is this going to change? Do we expect this to change in time or is this a fixed constraint? That's a great question. Real life. So I've been on Netflix for a long time, almost 10 years now.

[00:18:53]
And that means back in the day, engineering was a lot smaller. I think there's like 100 UI engineers, the entire company, it's a lot bigger now. But when Netflix was small, and this is before we even hit 100 million people, we only had three basic plan types, you know, we had basic, we had basic plus like sharing, and then we had premium, and these were hardcoded constraints that someone baked in really early.

[00:19:18]
Do you know how much effort it took to move away from that system, because there's so many dependencies based on the assumption that there's only 3 plan types. It took a lot of engineering effort to do that. So if you ask these questions upfront. Just ask, hey, is this going to change over time with the system? Is that something I need to design around? You're automatically making your system more scalable from the beginning.

[00:19:40]
But when you assume things, you're automatically constraining yourself and it's going to cost you more down the line. So that's a great question. Easy, right? Ask questions. All right, we're going to do some real examples now. And we'll see that it's tricky when you're in the moment. Again, you don't have a lot of time here. What's your instinct? What's your strategy? Because you got to get these down and you got to scope these, scope the requirements really quickly and write them down and then get that buy in so you can move on to diagramming.

[00:00:00]
So we want to focus on what the user needs to do. What is the core functionality? There's a lot of things a banking app can do and a URL shortening service could do. But we're not going to do all of them because that would be unreasonable in the time frame we have. So what do users expect us as a minimum? What's the MVP?