Enterprise Integration Patterns

[00:00:00]
>> Josh Long: I wanted to spend a brief moment here talking about enterprise integration patterns. And the reason I think about this stuff a lot is because increasingly, as we move forward in time, the body, the surface area of things with which I cannot talk grows bigger. There's a whole ecosystem here around enterprise integration patterns.

[00:00:15]
Usually this stuff comes from the world of like legacy 90s software, you know, where you've got like very, very hard, isolated things with which you want to integrate. But now everything is a distributed system. And so this technology, this idea, this practice has become more useful than ever.

[00:00:31]
What has changed is the idea of the broker, of the messaging queue as the hub around all around, which all of this sort of rotates, right? That just isn't a thing anymore, right? Nowadays you have your applications, those things need to talk to other things and so they hang off integration functionality.

[00:00:46]
So we could take that example that we looked at earlier, and if you needed to listen to messages coming in From Kafka or RabbitMQ or whatever, you could just add this method and be done with it in much the same way as you did event listener with the Spring boot events.

[00:00:59]
But what happens if you want to need to do a little bit more than just take a message straight off the wire and process it? This is where you get into the realm of enterprise integration patterns. And so we have this amazing book written by Bobby Hope and Gregor Wolff.

[00:01:14]
Bobby Hope was one of the people in the design patterns world back in the 90s, right? And it changed the world. And this is actually how I got into Spring is because of this book. This book was written in 2004. I was already using Spring. But I mean, when I say got into.

[00:01:27]
I mean, I started writing code for right. To contribute to the spring projects is because this book kind of changed my life. I was already dealing with Messaging Technologies circa 2004, 2005. I was using Rendezvous, I was using web methods, I was using Rendezvous from tibco, I was using web methods, I was using X Way integrator, which is not my friend, we didn't love each other.

[00:01:51]
And then I started realizing, okay, these things are very hard, sort of isolated things and they require a lot of consultants. You know, same thing for MQ series from IBM. They really wanted a lot of consultants. Biz talk I don't know about, I bet it's fine, I'm sure it is.

[00:02:06]
I don't know. But the point is I was really looking for something that would give me some of this integration capability, but that didn't require 10 consultants and golf games to get the job done, you know, So I look for open source options and there are some open source enterprise service buses right in the Java community.

[00:02:24]
Yeah, they even mentioned a whole bunch of different open source ESPs. Apache Camel, Mule, WC02, Oracle Service Bus, ESP fuse, Sonic Mix, Sonic, ESP, etc. One of these, I think it's Sonic, there's a guy named Dave Chappell, not the comedian, right? [LAUGH] Dave Chappell, not the comedian, I think it's spelled basically the same.

[00:02:45]
Dave Chappell wrote the original O'Reilly JMS book, right. And that book. You know, I didn't, I didn't love the technology, but I appreciated what this guy was doing because I think he was working at Sonic at the time and he did some really good stuff in the messaging space.

[00:02:58]
So I actually got him to sign a copy of that book and he, he signed the message. He signed the book. Get the message, exclamation mark. Anyway, all that to say messaging, super good, super interesting. There's a whole world of stuff around it, around how to process messaging, how to deal with it.

[00:03:17]
And there's four different types of integration styles that sort of naturally lend themselves to this discussion. File transfer. This is where one system produces data that gets written to a file that another one consumes. You've got shared database, in which one system writes something to a database and then another one consumes it.

[00:03:32]
That's a bit of an anti-pattern, right, that's a very un microservice. Today you've got rpc. Why is RPC a problem? Well, you're making invocations of methods on a remote service synchronously. What happens if that service disappears? You're hosed, aren't you, right? So it's the worst of possible integration options, but it's very common.

[00:03:49]
And then messaging, Messaging gives you all the benefits of all these. You're not coupled in that you don't have to be in the same place. Like files. I can write a file and then disappear and you can read the file later on. So it's asynchronous. And all they have to agree on is the location of the file.

[00:04:04]
Shared database has the same kind of benefits of file transfer. And then RPC requires you to be in the same place and at the same time. So these both require being in the same place. Messaging requires you to know about the same broker, but you can be asynchronous.

[00:04:19]
So this gives you all the benefits of these three things without any of the drawbacks. So messaging is a asynchronous by default, you can do things like fire and forget you don't have to operate at the same time, I can make a request and you can produce the response five hours later, right?

[00:04:35]
So it gives you a lot of decoupling. So by default, when you think about decoupling and building integrations in systems, you've got to think about it in terms of messaging. So I've shown you how to do REST and web stuff, right, the hypermedia and all that. But the final form of a big distributed system is going to be messaging, right?

[00:04:53]
You'll start doing it in terms of I send a message and that updates the state, as opposed to I ask a system to give me the state, right? And so I really, I hope you'll like, keep messaging in your heart. And I'm going to show you a quick framework here called Spring Integration.

[00:05:06]
Yes, sir.
>> Speaker 2: Is there anything that does, like state management of messaging? Like, so it's like a process of multiple messages that completes to do some specific task.
>> Josh Long: Did you look ahead?
>> Speaker 2: I have not.
>> Josh Long: Did you cheat? That's what we're talking about right now. Right now.

[00:05:24]
That's what I was about to say. Spring Integration is the project that does exactly that. And it has the ability to support all these patterns. See all these patterns on the right here? So this is where my story finishes up, which is I read this book, I'm a big fan.

[00:05:37]
And you know who else read this book? A gentleman named Mark Fisher over on the Spring team, right? Mark Fisher, he's a genius that makes terrible mistakes. And the terrible mistake that he made was he hired me, right? Which is regrettable, I'm sure. And he built this technology to support these patterns in an open source way in a nice clean, idiomatic spring way.

[00:06:01]
And so the API elements in Spring Integration map one to one to the concepts in this book, as opposed to what so many of these other technologies do where they have different names for what's already. This is the lingua franca of integration, these patterns. If you understand these patterns, then you're ready to go, right?

[00:06:17]
And if you understand these patterns, you can start writing code in Spring Integration. So I'm going to show you a very simple example using Spring Integration. And I've just shown you how to do Kafka. But we're gonna keep it much more simple here, right? Remember, in most integration flows, you've got a thing that produces events and then a thing that consumes those events.

[00:06:38]
So a source and a sync, right? In a messaging world, the thing that consumes the events from the external world is an inbound adapter, it takes the results of some external thing, be it a new file that appears in a file system, a new message that appears in Kafka, a new message that appears in an email inbox, a new tweet that gets sent out, a new whatever, right?

[00:06:57]
Any event that happens in the real world, you have an inbound adapter that allows you to pull those things in and then deliver them to consumers. Those consumers sit in a pipeline, like you were saying, right, things that process these messages, and there are components that act in that pipeline.

[00:07:13]
There's things for filtering, for transforming, for routing, for splitting, for aggregating, et cetera. And then finally, when you're done, you write the thing somewhere. You write the results of that processing out somewhere, usually. And that's an outbound adapter. So you might take a data. You might take, as we're going to do in this example, we're going to read data from a file system.

[00:07:31]
As new files arrive, we'll listen for the files and we'll start reading them as they get dropped into a folder, right? And then we'll process them and then, I don't know, write them out to the console or put them out. You could put them in a database, you could send them over Kafka, you could do whatever.

[00:07:43]
The point is, the code in the middle is completely unawares of what you did at the end and what you do in the beginning. It's a pipes and filters architecture, right? So the inbound adapter is the only thing you swap out to change the behavior of the connections, right?

[00:07:57]
The code in the middle is unaware of the inbound and the outbound adapters that proceed and succeed it.