Check out a free preview of the full Building Custom Data Visualizations course:
The "D3 Shapes and Layouts" Lesson is part of the full, Building Custom Data Visualizations course featured in this preview video. Here's what you'd learn in this lesson:

Shirley points out key portions of the D3 API that assist in data preparation, layout calculation, and user interactions. Applications of these concepts created by Shirley (and others) are demonstrated to help the audience understand the process of understanding in what way the data is best represented before approaching the D3 library for a solution.

Get Unlimited Access Now

Transcript from the "D3 Shapes and Layouts" Lesson

[00:00:00]
>> Shirley Wu: And of course the D3 API, so yesterday I went through this API a little bit also. But I want to talk more specifically about some of these sections that are relevant to us today, in terms of calculating layouts and translating that design, those data, into X, Y positions, into SVG path streams, etc.

[00:00:25] And so I think of the first two columns that I call data preparation and layout calculation. I think of that as all of the modules that help me translate raw data into what SVG needs to draw onto the screen. And whatever the output of those layout calculations, those are what I am plugging into that third category of I call it DOM manipulation.

[00:00:54] And that's the section of D3 that helps you with data selection, data binding and exit, kinda what people think about when they think about D3. So taking the output of this section and I'll put it into the third column to help me draw the SVG elements. So I want to concentrate on those first two columns because these are the columns that will help us translate that design into the code.

[00:01:26] And sometimes when we have a design, all we need are actually the scales down there to help us get from our raw data into screenspace. Sometimes maybe with the bars and the rectangles we had. Sometimes all we need is to go from each of those data to roots and whatever the channel it happens to be, X, Y positions, colors, sizes, etc.

[00:01:49] Sometimes all we need are scales. But other times we might want to put the data into these layouts and you might need calculations, specifically calculations that D3 already shifts with. And so these layout calculations will output x/y positions for us. So some of these are like, D3 will help you with histograms helping you calculate, kind of the height of each of those bars or the X, Y positions, sorry the X positions of each of the bins.

[00:02:36] It will help you calculate the X, Y position of a network diagrams and they use this thing called I think it's like force layout algorithm and it helps you. If you give it an array of nodes, an array of links, it will go and calculate X, Y positions for each of the nodes laid out in such a way that usually you'll specify for them not to overlap with each other, or keep certain distances away from each other if they're linked.

[00:03:16] Or repel themselves apart if they're not linked, etc. And so D3 force is one of my favorite modules just because it shifts with so much functionality that's so versatile. Sometimes I won't even use them for node and link diagrams, these network diagrams. One of the examples of how you can use force is these bee swarm plots.

[00:03:41] If you want to be grouping individual elements, these individual marks, these are points. Individual marks into kind of I guess-
>> Speaker 2: Those would be groupings of the graph nodes, right? On the bottom [INAUDIBLE]?
>> Shirley Wu: In the sense that it just graft across whatever this X axis is, I think they look like something that's quantitative.

[00:04:08] So some sort of a quantitative data attribute and then you can give it an exposition and you can just tell it don't overlap with each other, but it will kinda group around that quantitative attribute. And so D3 force has really great function, or a lot of really great functionalities.

[00:04:32] So that's one of the ones that I recommend skimming through the documentation. And, of course, the hierarchy section of the layout calculation, are really great also. They'll help you draw trees, or they'll help you calculate X, Y positions for each of the notes in the trees, it will help you with tree maps.

[00:04:57] Partitions and circle packs. So those are some of the possibilities. And then there are some layout calculations that will help generate path commands for you. And so these are strings, so for example, chord will generate the path command string to draw each of these chords. And so that's probably an arc command with a curve command and another arc and another curve.

[00:05:33] And so you don't have to kinda figure out how to write these curves. You can just give it your data, I think you can just give it your source target and start angle, end angle I think and then it will generate the string you need to draw this path in SVG, so that's really convenient.

[00:05:58] Of course it also has geo capabilities so if you have like geo JSON or TopoJSON that has kind of the geometry of each of these, I think these are counties. And so D3 would convert from, usually it's an array, I think. I don't do geo as often, but I think it's an array of latitude and longitude points that make up kind of the shape of each of these counties, or each of these countries or states.

[00:06:32] And then D3 will help you map from that latitude and longitude to an X, Y position on the screen. And then shapes helps you draw like archs so archs are a lot of times, you'll see them be used for like pie charts and donut charts, lines will help you draw line charts and areas will help you draw area charts.

[00:06:57] So a lot, a lot of options, a lot of ways that D3 will help you calculate layouts, and help you calculate paths. And I just want to go through maybe one of them. And I think a D3 port is really intresting just to show you how to look into the documentation and figure out what is needed as the input to give to D3 and what is needed as output.

[00:07:27] So for example, for chord and the chord is kind of, this one that we saw, so to draw one of these chords. So if we want to get the path string, we want to use d3.ribbon, and in that case, it expects an object that has within it a nested source and target and then you give it a start angle, end angle, and radius.

[00:07:58] And then you pass that into the D3 Ribbon function and it will give you back the string that you will then plug into your SVG path that will help you draw that. And then there are some functions in D3 that will help prepare your data for the layout calculations to draw into SVG.

[00:08:28] [LAUGH] So some of these are, this chord, D3 chord. This, let me show you right now, we'll prepare you. So sometimes maybe you don't want to manually calculate this data structure yourself. There's a helper function, that's D3 chord that's just if you give it an array of arrays, it will give you back that exact data structure you need to calculate those ribbons.

[00:08:57] And so if you just give it that array of arrays, it will give you back the source target with the star angle and angle and value. So that's extremely helpful so you don't even have to do that part of the data calculation yourself, you just need to give it your data.

[00:09:17] And so the similar things happen with D3, so nests help you prepare your data for D3 hierarchy, layouts that have hierarchy. And of course, hierarchy also helps with that and there's also pies that help and get your data ready to plug into arcs that will then generate a path string to draw onto the screen.

[00:09:48] But the reason why I wanted to share all of this with you and go into so much detail is, first of all, I think is really great to kind of just know all of the different layouts that D3 offer you. So that you can use them if you want to make a core diagram, if you want to make a tree, if you want to make a network diagram.

[00:10:11] But the most important reason why I want to share these with you is so that you know how to use them in an unconventional way, and by that I mean there is the examples that D3 gives us. And so kind of like the prescribed examples of how they recommend that this function is used.

[00:10:32] But then, if you can think about how, given this input, I get this output, so how can I manipulate that to help me calculate the layout that I want? And so I'll go back to the forest diagram example. So the reason why I think this is so flexible is because I'll often use it in ways that aren't in their examples.

[00:10:56] And I'll use it sometimes to just calculate and make sure my notes don't overlap. And sometimes I'll use it as just a way to have nodes bounce around on the screen. There was one time I did this for [NOISE] this visualization where I took all of the lines in Hamilton, the musical Hamilton and this adds absolutely no value to the data visualization itself, the fact that it bounces around the screen.

[00:11:34] But it gets that reaction. People see and then they're like, this is really fun. And my goal for it was to get people to keep scrolling down the screen and motivate them to do that. And this is all done with the force layout, and I just let it run a simulation where they bounce off of each other every time.

[00:11:58] And every time the reader scrolls, I let them rerun the simulation. But that's not one of the ways that D3 encourages you to use the force layout. I read through the documentation, and I was like, I can use it in this way. And, again, this is also the force layout where I grouped each of those lines by the song that it was from, and so for each song, I calculate it an X, Y position.

[00:12:34] And then I told D3 force that I want all of these news to go towards the group, the song that they belong too the X, Y position that they belong to. But I don't want them to overlap each other at all. So give them like a two pixel buffer in between or something like that.

[00:12:54] So go towards this X, Y position but make sure don't overlap. And this is how I was able to get a really nice layout of all the nodes grouped by their song. And this is like a variant of kind of the beast worm plot that I showed you earlier.

[00:13:14] I mean it's not one of the examples that's shown but it's not a far leap from those examples either. But I just want to show you these examples to encourage you to kinda, I guess the cheesy way to say it is think outside the box [LAUGH] with these layouts, kind of think about the possibilities there are.

[00:13:42] Another example I want to show you is, so Nadieh did this thing, I think [SOUND] she ended up calling this layout I think Loom, I can't remember. But you might recognize this layout as very similar to the quart diagram we saw earlier. But what she wanted wasn't just the quart diagram.

[00:14:11] Where a quart diagram has a source and a destination. What she wanted. And this particular visualization, this is a visualization of Lord of the Rings, the movies. And it's the characters and I think it's all of the places in Middle-Earth that they had lines. And so what she wanted was like she wanted all of the characters to be in the middle.

[00:14:40] And she wanted the places to surround them. The places that had lines, she wanted to surround them. And so she was like, this thing I sketched out, it reminds me of a D3 chord diagram. So let me see if I can mess around with the code so that it helps generate the layout I want.

[00:15:06] So she ended up messing around with I think the chord ribbon function I showed you earlier. And she messed around with it, messed around with the path strings, the curve commands, until she got to this point. So what I would encourage is like when you sketch your ideas, I knew that I said to look at different parts of the D3 examples and D3 gallery to get your inspiration.

[00:15:38] But oftentimes I would encourage you to, once you've seen enough visualizations, once you kinda just let yourself on the daily kind of just be on the lookout for new visualizations that are put out there and let yourself follow those and kind of think about, this visualization did this part really well, or this visualization didn't do this so well.

[00:16:03] After you get a good sense of the kinds of visualizations that other people are producing, when you start sketching, don't look at any other places. Don't look at the D3 gallery. Just like sketch out whatever idea is in your mind that you think will best showcase this data.

[00:16:23] This one, like she didn't go through looking at the D3 gallery being like, I think a quart diagram would be great to show this movie. She's like, I really wanna show where people are in Middle Earth when they have their lines and so maybe these other shapes I can have.

[00:16:42] And maybe these are some other ideas I have. And then once you sketch that out, she was like, that looks like a quote diagram, and she actually has a write up of how she started adjusting the algorithm for D3 ruben to come to the shape that she finally got.

[00:17:10] So I highly, highly encourage you to, when you're doing the design, just design it based on what you think will reflect the data best or the message that you want to communicate through the best. And then go back into D3 or go back into any other layout libraries that you might know and see if you can adjust it to fit your needs or or even if you have to rewrite it from scratch or like write some sort of layout algorithm from scratch, maybe that's what you need to do, yeah, creativity.

[00:17:48] And then, finally, I want to share with you some of the data visualizations specific interactions that I'm a huge fan of. These are parts of the D3 library. I will come back to them very often to do specific data visualizations, specific interactions. Like being able to brush through filter, being able to drag things around using Quang Tri, to find the nearest point quickly.

[00:18:13] Given an X, Y coordinate, using Voronoi to be able to do hover interaction on really small little points, or being able to zoom a pan, a map. There's a lot of choices here also that once you've kinda figured out your layout and you've kinda drawn your things, to add that extra layer that these are things that I come back to quite often.