Event-Based Integration

[00:00:00]
>> Josh Long: So that works. But don't you see the problem with this, right? We've got a Java object that we've injected into our controller and whose result we need to happen, but we don't depend on the result. In fact, it's void, right? Dr.schedule is void. Why do I do that, right?

[00:00:16]
Like why do I need this dependency? This is a problem because it means that this controller, this service, rather starts to run the risk of becoming overly complex, overly muddled with, with too many distinct concerns that have no business being in one type. So I want this, I need this to happen.

[00:00:30]
I need the scheduling of the appointment to go see the vet to happen. But I don't want that code to complicate this controller. It's a different concern. Okay, so let's back out that change. Let's think about how we can do this instead. And we'll do that. We're gonna back that out.

[00:00:44]
Okay, I'm not going to inject the dog tour there. There we go. And I want to move to a different way of integration. I'm going to use asynchronous event based integration. We talked about this a lot so far, but again, I think this is a much more natural way to sort of build systems.

[00:00:58]
So this is a great blog by Martin Fowler from 2017 before the COVID pandemic or BC. And in this blog he posits that there are four different kinds of messaging and integration styles, right, that are really interesting that we should all know about. The first of which is event notification.

[00:01:15]
And here you're using an event to tell one system or one module about the fact of a change, that something has changed, not what, right? And so, this is a very common example, right? Like if you're using like JavaScript. I'm sorry, but in the onload listener, there's only one onload, right?

[00:01:35]
I mean, ignoring frames for a moment here, right? There's only one really interesting onload. So when you say onload, you don't need to inspect the event to know what document was loaded, it's just the one, right? That's an event notification. The fact that the event listener got called is fact enough to give you actionable details about what's happened.

[00:01:53]
Okay, but sometimes you need details. And in that case, when the event gets sent from one module to another, from one system to another, it's called event carried state transfer. Event carried state transfer is the idea that each message, each event has actionable details that the consumer can use to then proceed and work on something.

[00:02:12]
And this avoids coupling the consumer with the producer because I don't need to know the origin of the message to query it to ask for more details, because all the details I need are in the body of the message itself. So event carried state transfer, very useful. If you communicate all detail changes, all system changes in the system via event carried state transfer, then something very interesting happens.

[00:02:34]
What becomes a source of truth in your system? If you have postgres, even in a so called monolith, modular, monolithic application, even in that scenario, you still have, it's very, very reasonable to have a lot of different databases, right? So you might have Postgres for your data for your writes, you might have Redis to support constant time read access, you might have elasticsearch to support full text search access of the data.

[00:03:00]
So these are two different, these are three different data structures optimized for read and write modalities, right? And in that case, what's the source of truth? What single database do you need to be able to reconstitute everything else, right? If you lost Postgres, could you reconstitute what's in postgres from elasticsearch or Redis?

[00:03:18]
Probably not, right? Likewise, could you reconstitute what's in Redis from elasticsearch? Probably not. You get the idea, right? The only thing you can reconstitute the entire system with is the messages. As long as you've got all the messages that were ever delivered to the system and you can replay them in the order they were delivered and you can rebuild the state of the system at the time that you, that you lost everything, right?

[00:03:39]
So you can RMRF your Postgres, your elasticsearch, and your Redis. And as long as you've got the messages, as long as you can replay them, you have everything you need to get back to where you were. And that's called event sourcing. And it gives you an audit log as well.

[00:03:52]
It's very, very powerful, right? So let's say you have some like Sam Bankman fried thing where you're like, you've got a little back door in the thing and you've got derivations happening in some profit thing, right? What's the office space? Remember that office space scheme? Like a fraction of a penny every transaction.

[00:04:09]
Well, okay, let's say they finally uncover the thing, right? Okay, this happened two years ago. Fine, go back to when the thing was introduced. In the code replay, null out the values, replay the events ever since then with the new algorithm in place, to arrive at the correct derivation Today you can do that because you have all the messages.

[00:04:28]
That's called event sourcing. And there are data structures that are really good at preserving every message you've ever gotten in order. In fact, for smaller systems, you can even use Kafka for that kind of thing, which is why it's kind of nice. You could also use a SQL database for that kind of thing.

[00:04:42]
Again, it's just a matter of preserving history. If you understand the example I just gave, we have read and write modalities, Redis, Elasticsearch, Postgres, etc. The idea that there are different data structures optimized for different kinds of read and write modalities, that's called CQRS, or Command Query responsibility Segregation.

[00:04:59]
Okay, all of these patterns, if you understand them, congratulations, you're an architect. Collect your promotion at the door. These are all part and parcel of messaging. So again, we've talked about messaging before, but remember, there's a whole world of things you can do with them even if you're in the same dayvm.