Transduction

[00:00:00]
>> Kyle Simpson: So let's think about how it was nice and easy if we wanted to compose three maps together because their shapes are compatible there using our equational reasoning. And we know they can be interchanged and that means that they can be composed together. But let's talk in a broader sense about other things that we might wanna compose together.

[00:00:20]
For example, filters and reduces. If we had a chain of several maps and a few filters and maybe a reducer or two, how could we put those together? And the answer to that question in transducing. This is a much more complicated topic. And I wanna warn you right now.

[00:00:37]
Of all the things that we've talked about in this course, this is going to be one of the most challenging to wrap your brain around. It's something that I've spent months trying to come more to terms with. And it's still something that I may say something wrong or get it backwards or something.

[00:00:53]
It still kinda sometimes trips me up. I'm at least comfortable enough with it that I can talk about it. But it is definitely an advanced technique. Or at least it's an intermediate technique that builds on the basics that we've been covering in this course. So the way I'm gonna present this to you is actually to show you the end result first.

[00:01:13]
And then in sort of an advanced way, we're gonna talk about how that solution is derived. And the reason I'm gonna do it this way is because what you can actually do is skip over the derivation. From here on out. It is not necessary for you to fully understand all the mechanics of transduction, to be able to use transduction to your benefit.

[00:01:34]
It is not something that I would encourage you actively to ignore. I as a teacher want you to learn all of these things. But I also want you to be able to practically use this stuff and you don't have to become a mathematician to use and benefit from these things.

[00:01:48]
So you're gonna see an API called transduce that allows us to take advantage of transduction. And you will see what Lego pieces you pass in to make it work and then we will peak under the covers and we'll see how potentially some mathematician came up with this crazy thing.

[00:02:05]
But once we've seen that, I don't want you to feel like you're gonna have to go through all those mental hoops every time you wanna use it. Essentially, pattern match if you see a map, a filter, a reduce, or any combination thereof, that you need to compose together.

[00:02:20]
Think about your mapper, your filter and your reducer. Those three functions are incompatible shapes. Map and filter are incompatible shapes. Because map has a single value in and a single value out. But filter, a predicate, is a single value in, Boolean value out. That's a different sort of a thing, it's a different kind of a mapping.

[00:02:44]
You couldn't take a value and have it propagate through a filter and have the value come out on the other side, it changes its nature. So you can't really just take a filter and a map and compose them together. And you certainly can't compose either one of them with a reducer, which takes two inputs.

[00:03:01]
So if we ended up in a scenario like this where we had a map, a filter, and a reduce. And I don't really care about the .map, .filter, and .calls themselves. What I care about is the add1, isOdd, and sum. Those three are incompatible shapes and I would like to be able to compose them together.

[00:03:20]
But I can't see how to do that readily with just my regular compose utility because of their incompatible shapes. So what we're gonna talk about this transduction technique is a mathematical transformation of those functions. It literally reshapes those functions so that they become a compatible shape that can be composed together.

[00:03:44]
In other words, transducing is composition of reducers. Transduction, transducing is composition of reducers. In other words, we don't wanna compose a mapper and a predicate and a reducer. What we wanna do is turn the mappers and predicates into reducers. And then we want a mathematical transform that allows reducers to compose together.

[00:04:15]
And that is the magic that this tool we're about to look at is gonna give us this API that's been built. That's the magic it's gonna give us. We're gonna show you under the covers how you will do it manually and talk about why that's a terrible idea to do.

[00:04:27]
You could manually put all of these three together into a reduce. You could manually say I'm gonna first add it and then I'm gonna call the if statement with the predicate. And then I'm gonna do a reducer to compute the new total and that could be your reduction.

[00:04:44]
And this is showing you again, the power of reducers, which is that they are this. They are this powerful that they can literally do mapping and filtering and reduction all at the same time. But what does this, what do the lines 9 through 13 shout out at you in plain and obvious language?

[00:05:08]
>> Speaker 2: Imperative code.
>> Kyle Simpson: Imperative, that is the imperative approach. So yes you can do that. And it would probably be better to do that than to do nothing, especially if you have large data structures. But this isn't the functional way of doing it. We don't wanna come up with imperative solutions.

[00:05:25]
We want declarative mathematical solutions, and that's what transducing is, okay? So instead of doing that, we don't wanna take that approach. Instead of doing that we want to use transducing. Here's how transducers works. We have a compose function, which is exactly the same as the Compose that we've seen.

[00:05:44]
There's literally no difference. It's not a special compose. But notice what I'm composing here, I'm not composing add one and is odd. I am composing this thing that has come back from these two different little utilities, map producer and filter producer. Those are utilities which will be provided to you by your functional programming library that supports transition.

[00:06:09]
So you don't have to write those utilities. You just have to use them. And the way you know how to use them is if you have a mapper, you pass it to a thing called map producer. If you have a filter, you pass it to a thing called filter reducer, and if you have a reducer, you don't need to pass it to anything, because it's already a reducer okay?

[00:06:27]
It's a mathematical thing that is essentially reshaping the map, into a reducer, or a mathematical thing that is shaping the filter, into a reducer. The way it does that's kind of magical. And we'll talk about it but it's reshaping them into reducers. A special kind of reducer, not exactly the same kind as you might expect.

[00:06:48]
It's a special kind of reducer that is composable. So how do we put these together then. Because you notice I didn't list some in the composition. That's because those, the result of MapReducer and the result of Filter Reducer are a special kind of reducer, which is, it's not really ready yet.

[00:07:10]
It needs a reducer to be passed to it to become a reducer. And I know that sounds strange. But it's like in this prototype stage, it's like, I can become a reducer as long as you pass me a reducer. That's what those utilities are doing. That's the mathematical shape transformation that they're doing.

[00:07:30]
So we compose those things of that prototypical shape together. And that becomes what we call a transducer. So specifically a transducer is one of those prototypal shaped producers, which will become a reducer if passed a reducer. Okay, that's what a transducer is. So how are we going to use that thing?

[00:07:56]
We're gonna make that transducer, notice that I did not add in my final reducer. The reason I didn't add in my final reducer is it's the thing we're gonna pass in to make all of that into a reducer, right? I said a transducer needs a reducer passed into it.

[00:08:12]
So I'm holding some back. I didn't include it in the composition. The function itself is going to become the input to this composition. And that mathematical transformation then produces this composed reducer. So how are we going to use it? We're gonna call a utility called transduce. This again will be provided to you by a functional library.

[00:08:34]
And we're going to pass in a transducer which we've made. And we're gonna pass in a reducer, a combinator, which we have here, in this case, the sum function. We're gonna pass in an initial value, that's the zero. And we're gonna pass in our data structure, in this case, the array.

[00:08:54]
Those are the four pieces that transducers needs. It needs the transducers. It needs the reducer that is going to get passed into that thing to produce the reducer. It needs the initial value, and it needs the data structure. And what it does under the covers is it passes some in the transducer for us producing that new composed reducer.

[00:09:17]
And then it reduces the list with that new reducer.That's it actually. Actually all of the hard work was done by Map Producer and Filter Reducer. Transduce as a function is two lines long. It doesn't do much. It simply passes one thing into another and calls a reduce on it, okay?

[00:09:39]
So, There is another helper which makes it even easier. Another helper that is generally provided by this libraries and it's called into. You'll notice that into is taking our reducer, I mean our transducer, our initial value and our array. But what's missing is that we didn't need to provide it a reducer.

[00:10:01]
Because what into does, is it looks at the data type of the initial volume. In this case it's a number. And it says, I know what kind of combinator numbers need. Numbers need a thing that adds two numbers together. That's the general Combinator for numbers. So it automatically supplies us a thing that looks like the sum function.

[00:10:26]
If we had said into transducer and we'd given in the string, then into would substitute in, a string concatenation combinator. If we had passed it an array, it would institute an array push combinator. So it knows, based upon the data type, what kind of combinator, what kind of reducer is needed.

[00:10:47]
We don't even need to pass one in. So either what we do on line 10 or what we do on line 18 is transducing. It is running through the list, just once, and doing the mapping, filtering, and reducing, all at the same time.