Introduction to Data Visualization with d3.js v4 Working Together

[00:00:00]
>> Shirley Wu: I started using D3 and React together I think back in 2014, because my job started using, decided we wanted to move from Backbone to React. And one of the biggest problems,
>> Shirley Wu: Okay so this is, but everybody knows React and what React does? Cool. So D3, it has the enter-update-exit pattern and React has a virtual DOM diffing.

[00:00:29] It basically does, I think I say they both try solve the same problem, but they also, the problem is they both want control of the DOM. With React and a lot of those like MVC frameworks, like a lot of the problem is the fact that both of them want control of the DOM.

[00:00:47] About two years ago after we started using React, I went and figured out a few different approaches and I actually wrote about one of the approaches in my Medium article.
>> Shirley Wu: And it's called On D3, React, and a little bit of Flux. And it goes into extreme detail about how I use React and D3 together in such a way,

[00:01:22]
>> Shirley Wu: Yeah it goes into a lot of detail. Because I think it was a little bit of a non conventional approach. So I have three approaches and that's my second approach. And I think in this talk, I talk about like-
>> Shirley Wu: So, who should be doing the data calculation and who should be doing the rendering.

[00:01:50] So approach number one is the use case is an application with a pretty small data set and a pretty simple visualization. And in that case, I use React for the structures, so base structures I mean the component structures cuz that's kinda nice. And then I use it D3 for the data calculation and I use React for the rendering.

[00:02:15]
>> Shirley Wu: So,this is I think the example code I have, it's basically a force layout that when you click on update button it's just like randomly generates new node data and then updates through that. And this is built with React and D3 with my first approach where D3 does all of the data calculation and React does all of the rendering.

[00:02:41] So I believe this is, yeah, so I have a React's graph component, I create a graph component, and then I say okay, when,
>> Shirley Wu: And this is where I defined the force layout. And then I say, okay when the component mounts or will mount, and this is also version three, just to put it out there.

[00:03:12] This is version three of everything, of D3. So on each tick, force an update on the React component. Then on each update what react is doing it is literally drawing all the nodes and links. It has the r attribute, the x, the dy, the x1, y1, y2 of the line.

[00:03:45] So React is taking care of all of the rendering. And then, yeah I return all of the nodes and links. So this is the first approach. Yeah, so D3 is just doing the data calculation of getting the X and Y of each of the nodes. And then React does the setting the attribute on those nodes and links.

[00:04:12] So this is what I think is like the most straight forward simple approach. And so the pro of it is like it's very clean, you're like your data, or I mean sorry, your code and how you reason about it is very clean, it's very easy to reason about it.

[00:04:27] But the con is that [COUGH] you have to do the reacted life cycle on every single tick. So like it's literally going through, so react has the whole component I think, I guess like component should update. Component did update and all of those things in between that I can't remember and that's also where they're doing the virtual DOM diffing.

[00:04:53] And so on every single take and those takes are fired really quickly and that life cycle is been called for every single take, and that can get really unperfomant. And this is also true for things like animations is really hard for this approach. Using D3 transitions is hard, using this approach because with the D3 transitions, D3 needs access to the DOM to be able to update the attributes on every single one of those interpolations, the animation interpolation.

[00:05:34] So you can't use D3 functions that need access to the DOM. For example, the D3 axes that we used earlier, that's like super nice, we can't use that because that's D3 taking control of the DOM and rendering. You can't do anything that basically needs to update really quickly.

[00:05:55] So you can't do drag functions, because you need to, or I mean you can but drag functions can get unperformant, and zoom functions get unperformant. So there's a lot of things that if you, if all you need is a little bar chart, a line chart that doesn't need to update all that often or doesn't need animations or something.

[00:06:17] And you just need to stick that into your React app, this approach is really great. But if you want to get pass that, to like different interactions that need a lot of updating, it does not do as good of a job. So the second approach is the approach I mentioned earlier that I covered in extreme detail in this article.

[00:06:39] But the basic thing about it is, so the use case is an application with a really small dataset, but it's a highly interactive visualization. And the way that I think about that is I use React again for component structure. D3 for the data calculation and D3 and React for the rendering.

[00:07:05] So React takes care of the elements of entering and exiting the elements and then D3 takes care of updating the attributes. So my example code.
>> Shirley Wu: So this is exact same thing that we saw earlier, but the only difference is,
>> Shirley Wu: Okay so this that same graph component, and D3 is still doing the calculation on each tick of the x and y position and calculation.

[00:07:44] But now what we're doing is, we are creating a node and link component for each of the nodes and links, so that when component mounts
>> Shirley Wu: Wait, hold on, this is where I mean that it's been a while since I looked at this code. Okay, so I think, because I structured it this way.

[00:08:20] Okay, so componentDidMount will basically Map to the enter, like what you would do for entering a selection. And then componentDidUpdate, will map to the update selection, what you would do to the update selection. And then I think component will unmount and will be the exit selection. So I think that is all I'm doing here, that I'm calling the enter node and in enter node all I do is.

[00:08:58] So basically for the node component, react component, I define a circle and I define a text. But I set the r attribute here, I also tell it you can force drag, and I set the x and y of the text and update node as the force is ticking through it.

[00:09:31] I set the x and y of the node with D3, is the drag not working? Yeah, so this is like a lot more convoluted of an approach, but the reason why we ended up with this at the company I used to work for. One of the reasons that we moved to React was because when we had backbone.

[00:10:04] We started getting really, really confused about all of the elements that needed to be entered. And all of the elements that needed to be exited, and how do we keep track of them? And then like, on what event should we be entering this, on what event should we be exiting this, and on what event should we be doing the update?

[00:10:23] And all of this got so confusing, and we couldn't keep track of all the interactions that when we moved to React we were like. That's it, we're getting React to figure out what needs to be entered and exited because that's the hard part that we couldn't keep track of.

[00:10:37] And so we're gonna use the virtual dom to calculate that for us. And all we're going to use D3 for is updating the attributes because one of the things that we wanted was being able to animate. And so like I mentioned before with D3 transition, D3 wants access to the X and Y attributes.

[00:10:58] So if I just give the control of the attributes to D3 and have React do the enter and exit, then they're not actually overlapping in control. And so the thought process behind this works, and it worked well, and I think this is example app, and I think the whole point was if you have a lot of data.

[00:11:30] That changes or you can manipulate things cuz this one was I think, if this was a budget app like that example expense app that I made for this blog post. That then took me like two months because I was like, I'm gonna add this functionality, I'm gonna add this functionality.

[00:11:48] And I think all that I wanted to demonstrate here is there's different parts of the page that can edit the data in different ways. So I could delete this and then that expense goes away or I can add one that's like coffee and then that's like three dollars maybe.

[00:12:07] And then adds and then I can categorize it, and there's a lot of things I could do that can change the visualization. And I think all I wanted to show for it is the functionality and then showing you the different animations and transitions. And that's possible with this second attempt or a second approach.

[00:12:32] So the pros and cons, the pro I feel was it really took advantage of the respective strengths for React and D3. So D3 obviously, always use it for the data calculation but then we can take advantage of D3s transitions, but then we can also use React to do the virtual dom.

[00:12:59] So that we don't have to have the mental strain of personally taking care of what to enter and exit.
>> Shirley Wu: So the cons were the React component around all the nodes, if you start having lots of nodes, that's extremely unperformant also. Another con is if you notice with this example I don't actually have exit node because.

[00:13:30] Or I don't do anything with D3, I don't update any of the attributes for D3 on exit. You can't actually animate the nodes out with this particular approach because React will just immediately remove all of the nodes that should be exiting. So you don't have enough time to do your animation, you'll start your animation and will just be like remerged right away because.

[00:14:09] React in D3 don't have that communication to know that the nodes should only be removed after the animation completes. So that's another thing, and then it also made people really uncomfortable. There were like, this feels wrong, this feels wrong to use React in D3 like this. I thought about a lot cuz I was like, well maybe.

[00:14:35] But at the end of the day, if your product has certain needs and then it's not giving you bugs, like if this approach isn't giving you bugs. Then I guess, how do you define wrong, what's best practice? And so that's my response to that.
>> Shirley Wu: So the third approach is actually the approach that I pretty much use now.

[00:15:14] This is pretty much how I code my React in D3. So, the use case for this that I originally thought of was an app with a really large dataset and frequently updating visualization.
>> Shirley Wu: So for that one, I use React for the component structure and I use D3 for the data calculation and I use D3 for all of the rendering, whether it's enter, exit, or updating.

[00:15:39] And so all that happens there is essentially,
>> Shirley Wu: It looks uglier but all React does is make this group container, we set a ref on it so that D3 knows to. Wrap that node, wow, this is a really old version of React too [LAUGH]. But yeah, so it basically grabs that group element and wraps it in d3, and then we just D3 the whole way through.

[00:16:24]
>> Shirley Wu: The whole thing and, yeah?
>> Audience 1: So, on this strategy you used to react to just create a container.
>> Shirley Wu: Yeah.
>> Audience 1: And so here at d3, here is a nice tasty piece of a real DOM just for you.
>> Shirley Wu: Yeah.
>> Audience 1: And then now, is this, now you can go about using d3 just the way everyone here has learned it, where you can have transitions, you can have enter exit update, right, and all that stuff.

[00:16:48]
>> Shirley Wu: So.
>> Audience 1: Okay.
>> Shirley Wu: Yeah, basically, all React did is to react, that group is this black box, that it doesn't need to care about, and two, d3, it's like, this container that you can go crazy in and have fun in And so that's this approach. And then so back when I first gave this talk I was like the pro, the visualization scales, and you can use all the d3 functions, like you would just use d3 in any, you can even copy over Mike Bostock's index.html.

[00:17:20] Well I mean not all of it, but you can copy pieces of the code over and it works as long as you scope it correctly, and this is great except the con is then why are you using, like where are you even react both? And so I actually think, since I gave this talk a year and a half ago I have come to like really appreciate this approach, because it, one gives me the freedom to use d3 anyway I like.

[00:17:47] And then it also likes gives me the freedom to use react, like there's something that's amazingly beautiful about how react forces me to think about, like components, componetizing like my visualizations, and so I love that and I love using react for just communication between each of the different components.

[00:18:10] So if I have multiple visualizations that are linked together, like if this visualization effects what is displayed in this other visualization. React makes that like communication super easy for me whereas like I was just used d3 or something. I will have to use like a bunch of dispatches to like do the communication which is kinda like what we're doing with backbone reacts like makes that thought process super easy.

[00:18:37] And then, I think I also appreciate the fact that my application isn't going to just have visualizations, right? It might have, like for example, that Obama piece had a lot of text that explained things, my Hamilton piece had different parts to it also. And so I use React normally for all of those non-data viz parts of my app and use this approach for the visualization part of my app, and I just do whatever I want with d3 within it.

[00:19:15] So the lessons I learned was the most, most important, if there is nothing else that you get out of this is never, ever, ever have React and D3 controlling the same parts of the DOM, or else you're gonna get nasty bugs. I think I still very, very vividly remember, I think I had used I accidentally had React to keep track of an attribute or something or a specific element and then d3 do something to it, and I was just getting nasty bugs for the whole entire afternoon, then I was really sad.

[00:19:49] Just make sure their controls don't overlap and then I then don't think I've ever seen bugs as long as I don't have their controls overlap. Yeah, that one's cheesy, and then these ones assess the needs of the project and then figure out what the combination of d3 and React is that makes sense is, and that's Zootopia.