When to Apply Architecture Design

[00:00:00]
>> Erik Reinert: Now we're gonna go through the when. I actually really liked this GIF [LAUGH] because I felt like it was a good example of when you should do something, especially survivability, right? And a lot of this next section is really supposed to help you understand when you should do some of the things to build these designs or when or you should or when you shouldn't.

[00:00:26]
So now we're gonna get into when to use backend architecture designs. So again, we're gonna start out with a quote from AI. I did not know AI was as prolific as it is, but I'm here for it. When I asked about the how you would summarize this section it said the right time to act is not simply found, but felt.

[00:00:46]
It arises at the intersection of preparation and intuition where opportunities whispered becomes too compelling to ignore [LAUGH] And I loved how whimsical that sounded but it's actually very true, right? A lot of the times when you're trying to implement something, it might be a gut feeling of like, okay, this isn't gonna work.

[00:01:09]
I know this isn't gonna work, or it might be intuition. But a lot of understanding of that complexity. All those other things we talked about before and when to use them is also really important, especially when you're at a company trying to make large changes. Or even in an interview and somebody is, when would you do something?

[00:01:35]
It can be a really, really quick red flag to an interviewer. If you go like, why would you use this? And they're like, whoa, hold on, pump the brakes, man, like, we're not even there yet. And having that nuance can help you again get the job that you're looking for or help you make the change you're trying to make, yep.

[00:01:56]
>> Student: Yeah, I had to reject a candidate that I really liked because that was like everything. Yeah. The solution was ready and it was this tool.
>> Erik Reinert: Yeah.
>> Student: Absolutely.
>> Erik Reinert: Yeah.
>> Student: It's like, I really like you, but.
>> Erik Reinert: Yeah, it's scary, it's scary. You kinda look at it and you go like, do I trust this person?

[00:02:16]
Because I don't know. And like you and I have even talked about this before, Mark, with some of the like, just to go back quickly, I proposed the other course as well. The platform directory course, and I remember you were like, I don't know what this is, even accomplishing.

[00:02:33]
[LAUGH] And yeah, you have to, again, part of building these textures and and doing these things is making sure that you're moving with the company. Making sure that you are making everyone feel comfortable. That you're not trying to just overload what your goals are with maybe what the company's goals are.

[00:02:57]
And yeah having that ability to act when the time is right rather than making it your job to constantly change things is a much different nuance. Okay, so when to use backend architecture designs? We're gonna talk about understanding when, the factors of when, and considering when, right? So let's get into it.

[00:03:23]
So understanding when to use backend architecture designs. The first one probably makes a lot of sense to most of you, which is when starting a new project. Does anyone here know the term greenfield? What does that mean? Can somebody give it to me? Anybody?
>> Student: Just starting something brand new like from scratch.

[00:03:41]
>> Erik Reinert: Exactly.
>> Student: Not evolving an existing thing.
>> Erik Reinert: Exactly, yeah. So when you're talking about a greenfield scenario, you're talking about something that doesn't really have impact anywhere, right? Why is this valuable? Well, because you don't have, like you lose a or not lose, but you, you get rid of a lot of the challenges of, well, why should we do this, right?

[00:04:05]
How does this work? How can we make this integrated into this? If there's a completely like going back to the rules thing that that person mentioned before, right? It may be easier to implement that rule system when that rule system doesn't touch anything else, right? Or it's completely new and it's a completely new part of the system that's never really been explored before, or things like that.

[00:04:28]
A new project can help you have opportunities to explore, try new things and build new systems. Even again, Mark, what you were saying with the candidate, right? They might have looked at a lot of that as greenfield because they weren't here yet. They might not have understood all the already technical foundation you've created, right?

[00:04:55]
And so that could be the difference between somebody coming in and changing everything right out the gate. To coming in and actually integrating with those systems and working on growing what already exists. So greenfield can be a really good opportunity for that, but you have to know when that is actually a greenfield scenario.

[00:05:15]
>> Student: Yeah, I think a lot of companies are probably like generally broken and looking for a candidate coming in to solve like particular solutions. But there are cases where the system works great.
>> Erik Reinert: Yeah.
>> Student: It's scaling wonderfully and people are loving it, so it does happen.
>> Erik Reinert: Yeah.

[00:05:33]
>> Student: But I think that's, yeah.
>> Erik Reinert: Yeah, no, absolutely, having that understanding of when it's greenfield as well is really important. Again, if you come into something with a fresh mind, but you don't have the context, it might not actually be greenfield, right? And that can make you look bad, in all honesty, that means that you didn't do your proper research.

[00:05:59]
It means that you didn't think about all the already existing factors, right? And if somebody can come into a conversation and easily be like, well, no, this isn't new. [LAUGH] Like we've already got all the like, what about all this over here? And you didn't consider and be ready for that kind of discussion, then people are gonna lose face in what you're trying to accomplish or what you're trying to do.

[00:06:23]
So knowing when it's a greenfield scenario can be important, can be helpful, but having that context is really important. So the next one is understanding when it's scaling an existing system. Because system designs can help use to identify bottlenecks in areas for improvement. So again, going back to what we were just talking about, maybe most of the system works great, right?

[00:06:49]
Maybe everything's good, everyone's happy, we're making money, developers are working and they're productive. But you might notice one particular problem, right? And you go, okay, well, I'm not gonna scale my focus back out to this huge view. I'm just gonna focus on this one particular problem and try and build a good architecture or system design around that.

[00:07:15]
This is what I do a lot, to be clear, right? At the company I work at, a lot of the times, we've already got something in place. And more often than not, somebody at my company will be, well, what about this? What about that? And you will get to a point after being at a company and developing systems for years and iterating over that a long period of time.

[00:07:37]
Most of those problems might already be solved, right? However, say these solutions have also introduced unexpected challenges, right? A good example of this is, say for example, whenever you have a user sign up it sends an email, right? More often than not when people will create that, they'll put the email request in with the sign up request, which means that that request takes even longer.

[00:08:07]
Because now it's gotta go out to your email service, send that message, wait for that message to come back with a success. And so when somebody clicks sign up, right, they're now sitting there waiting for like maybe five, 10, 15 seconds. And just staring at a loading bar or something like that, or a loader, right?

[00:08:25]
That is an opportunity of a bottleneck, where it's to say, how can we get users in quicker? How can we get them using the platform faster, right? And maybe there's a way we can introduce things like asynchronicity, right? Or separating that logic out so that the business gets to get to where they want quicker.

[00:08:42]
And I'm gonna be honest with you, you will always be able to sell something if you can sell it to business. Engineering is always gonna have a high opinion, but at the end of the day, the people who literally control the money of the company are normally the people who will have the ending decision.

[00:08:58]
And if you should or should not do something. And so I've dealt with that a lot where I've wanted to make a change and then I've approached it from a technical perspective, right? And gotten it thrown away, but then gone back to it and been like, well wait, no, this also gets people onto the platform quicker.

[00:09:16]
It gets them more involved with the features that we want faster, things like that, then business will be like, yeah, yeah, absolutely. Yeah, yeah, we like that, yes, yes, please. So existing systems and figuring out where you can identify challenges to or problems to improve can be really helpful.

[00:09:36]
So understanding when there are complex problems that need to be broken into manageable parts. Going back to the original workflow or workflow I showed you of the Quark V1 infrastructure, that's a good example of that, right? For example, if I was to say, okay, we wanna use serverless, but honestly, I don't wanna spend the man hours of having a team manage the open source software, excuse me, the open source software that we used, right?

[00:10:05]
Going back to the question of when to use open source and when not, this is a prime example of that, right? Where we're spending 10, 20 hours of engineering hours a week when we could just pay somebody to take care of it. We could just literally have as a cloud provider, provide that as a service, and that is, that's also decoupling the complexity of a system and really just giving that complexity to someone else.

[00:10:30]
Saying you've got the engineers to do it, you've got the time to do it, so why don't you take care of it, and that can be a part of the architecture rework. Where it say like, all right, well, we're gonna rip this out. We're gonna have this go to here now, and then, there you go you have a new design, right?

[00:10:50]
Understanding when there is a need for improved efficiency and productivity, right? And a good example of this is understanding not just a bottleneck, but understanding where developers have multiple touch points or data has multiple touch points. [LAUGH] I had somebody in my chat one day come in and say that, [LAUGH] I can't believe I'm gonna mention this, that UUIDs are not scalable.

[00:11:18]
Can anyone guess what they meant by that?
>> Student: You run out of them eventually.
>> Erik Reinert: No, no. So when we're talking about like users, for example, right? If you do nothing but user IDs for everything, you're gonna have to need a lookup with that, right? And if you're in a distributed system, and you have to do multiple lookup, between services, you're doing the same request over and over and over and over and over.

[00:11:45]
And then you've got data or logic that has to be updated in each one of those systems over and over and over. And I was actually interviewing at a fang company, I'll put it that way, and that was a problem they were trying to solve. They were at a huge scale, but they constantly had to do lookups to user information for like login, authentication, things like that.

[00:12:07]
And that's what that person actually meant, but understanding, right? if you need that efficiency, if that is something that's a bottleneck to you or not, can be another opportunity of understanding when you can make that change. Maybe instead of having to do multiple lookups in multiple places, you're passing context, right?

[00:12:28]
So you have context that passes between services that gives more information about the user so there's less lookups and less stress put on the system. So efficiency, that also comes productivity, right? A good example of that is say developers, when they go to make a change to a system, they have to update three places, four places, five places, right?

[00:12:51]
That's even very common as well, because more often than not, developers copy and paste code. Avoiding that so that productivity can be smoother, whether it be, okay, well we know all the user stuff is here. Maybe we need a user service, right? Maybe we need a way so that the user all logic is all together and then we can have another service handling transactions or something else.

[00:13:16]
That is not just productivity for the infrastructure or the, what is it, not the infrastructure, the services and their scalability as well as performance, but it's also the performance of your developers, right? Having one team focused on one tool or one service, and that's all they have to focus on.

[00:13:34]
And that kinda steps into the service or microservice design that companies move toward. Because as you grow bigger and bigger, you wanna separate those teams out so that they have a narrow focus on what they're working on, and then those components get glued together easily. And this one's also really important too, which is when there's a need for better communication among team members, right?

[00:14:00]
Maybe there's something that got implemented that never was thought through or never got documented. Maybe there's already a system design that exists and is in there, but it's nowhere to be found in documentation, right? And this can be something as simple as going out, looking at that system, documenting seeing it wire, framing it out so that people don't have to depend on things like tribal knowledge or guessing.

[00:14:26]
And it makes it easier to understand the communication, especially for things like handoffs and stuff like that. So designs can be helpful with communication as well. And again, when you're talking about 100s of developers, 10s or 20s of teams, the power of just a simple document shared around is irreplaceable.