Backend Architecture Q&A

[00:00:00]
>> Audience: Are you gonna talk about managing complexity and simplicity? To give an example, I'm working on an email service that moderates encrypted-signed standard emails. I'm implementing a rule system, but the other developer doesn't know the system. The rule system adds complexity, but it adds flexibility, modularity, and scalability for future additions to the system, but adds complexity to the design.

[00:00:27]
How do we manage complexity down the road?
>> Erik Reinert: Okay, so if I'm understanding the question is basically, well, so, okay, so I think a good approach to that is planning. It's really around planning. If you have something that you know is going to be a big change to the system but it's for the good, right?

[00:00:52]
Your job is not to technically build that well, but it's to educate and to share the understanding of it well. You're not gonna be able to get that sold unless the other developer and any other person on any other team can confidently answer why you're doing that rule system.

[00:01:12]
You might think it's a great rule system, but if you don't have documentation that backs why this rule system should exist, again, where I work, at Hippo, we have technical documentation around anything. The first time anyone brings anything up in a discussion, it's okay, where's the TDD, right?

[00:01:30]
And that's not meant to be rude. It's meant to make it so that our culture focuses on education first, right? It's kind of like being in a meeting, but then not knowing the agenda, right? You kinda just sit there and you're like, okay, as long as everyone's on the same page, then you can make any change that you want.

[00:01:50]
But I do think what that person is dealing with is more of a communication challenge than the actual technical side of it. And so figuring out what works best for you. For example, at Hippo, we want to move to Kubernetes for everything. That wasn't an overnight thing. I started working on that a year ago, and I just showed a demo to the entire company last week [LAUGH].

[00:02:18]
So it took a lot of preparation, a lot of preparing, and also being prepared for anybody who might push back. Right, be prepared. Engineers have opinions, they all do, gosh, they do. And you have to be prepared for those opinions and be open to them as well. Don't be like, well, it's good system, just deal.

[00:02:40]
No, you need to be able to handle that in a way where you're actually trying. You're a salesman, you're trying to figure out how to sell what you're trying to make. And so I think with complexity also needs to come understanding.
>> Audience: Do you have any advice when to use open source solutions like Keycloak or making your own auth?

[00:03:07]
>> Erik Reinert: I can't say much about making your own auth, because I pretty much always do that [LAUGH]. I have tried a lot of different systems, I've tried Clerk, I've tried Auth0, I've tried whatever else. I think you need to be able to be confident in which system is going to give you the most value.

[00:03:29]
And that might mean, again, timeboxing it, setting requirements and asking, and even potentially exploring, right? Open source is great, but business is business. And if business can afford to pay for something so that engineers don't have to manage their own auth for five hours, eight hours out of the week, then you're saving engineering time.

[00:03:51]
And I guarantee you, the cost of the engineering in a company is normally a lot more than the resources you put towards cloud infrastructure or something like that. So a good example is, if you wanted to go with an open source solution, make sure before you make that decision, that you understand the maintenance challenges.

[00:04:11]
Make sure you understand the scalability challenges, because at some point you're gonna have to figure out how to maintain, or how to update that system. You're gonna have to figure out how to fix it if something breaks. And if you're really good at that open source framework, then yeah, it might be a perfect fit.

[00:04:31]
But I think checking all the boxes on the requirements, going even back to, how do I prepare for future complexity? That other question fits really well into here. Because you might use an open source project and solve a problem really quickly. But then down the road, you didn't expect the complexity of, for example, Vault uses a Raft consensus algorithm.

[00:04:56]
How many here know what a Raft consensus algorithm is? Okay, okay, one person, yeah, there [LAUGH] you go. Do you as well? Okay, so yeah, so one person. So if you don't understand how it works, but you just deploy it because it's easy to deploy, it's not the same as making it an educated decision on whether or not it should actually be used.

[00:05:22]
>> Audience: When you refer to documentation, are you always referring to markdown files or things in that nature? Or sometimes TDD or tests and stuff for this documentation?
>> Erik Reinert: Any kind of documentation. Yeah, any kind of documentation. One of the best pieces of advice I can give anyone out there listening is, learn how to become a salesman, right?

[00:05:46]
Sometimes it means flashy charts and things that people can get excited about. Sometimes it means very verbose documents in Confluence that are just paragraphs of you writing and getting all of your thoughts on the paper. Both are valuable, both have their use case. But I think it's really, you have to think about your audience.

[00:06:07]
And that was a big challenge for me when building systems like this, too. Was, it's like, if I'm talking to developers, flashy, flashy, [LAUGH] right, cuz that's what dev's like. Sorry, [LAUGH] but it's true. Whereas stakeholders, one of the biggest challenges I had was writing huge documents and then going and being like, okay, let me show this to a business owner.

[00:06:27]
And them being like, I'm not gonna read this. I don't have time to read this, this is huge. This is a thesis on why you should use Kubernetes, right? Break it down into bullet points, get it to me so I can read in five minutes. And different documentation solves different problems.