Create Amazing Graph Visualizations Using D3.js

Note to reader: If you don’t want to wade through the technical details of what I am about to do, you can see the final visualization and play with it at the end of this post here or at my Github pages here. The visualizations in this article are designed for viewing in a desktop browser and may not render properly on tablets and mobiles.

In a previous post I demonstrated how easy it is in the igraph R package to create networks and detect communities within those networks. I used a publicly available edgelist which connected characters from the Game of Thrones TV series based on scenes that they appeared in together over the eight seasons of the show.

In that tutorial we saw some interesting static charts which demonstrated the overall network and color-coded the eight communities that were detected. Here’s an example of one of them, with the communities color-coded and the most connected character in each of the six major communities named.

Now I want to extend this to see if I can develop a highly responsive web-based visualization. If you want to see the end product, scroll to the end of this post. What I want to do is:

  1. Watch the network form an equilibrium using a force-directed algorithm
  2. Allow a user to grab a character and move them, let them go, and then watch the character find their place again.
  3. Color-code the communities.
  4. See the name of a character on mouseover.
  5. Append photos for the most central character in each community.
  6. Create a legend for the communities.
  7. Allow the user to adjust the network so that it will only connect characters based on a certain number of scenes together.
  8. Create a search function where the user can type the name of a character and all other nodes will fade out temporarily so that the user can see precisely where the character is.

This is a lot of customization, so we will need to move into Javascript’s D3 library to do this. D3 is an amazing Javascript plugin designed to create data-oriented visualizations. It’s not easy to learn Javascript and D3, but if you are willing to put in the effort it can open up a whole new universe of what’s possible.

I am not intending to teach you Javascript or D3 in this post. But by outlining my steps and process, you might be able to pick this up and do something similar once you’ve mastered a little of those languages. To make this whole process easier and to save time, we can prepare all the data we need in R, and we can then lift some existing D3 code and customize it to do what we need.

Preparing the data

At the end of the last post, we were working with an igraph object which contained all of our nodes and edges for the the network based on a download of the edgelist from a Github repo. We gave each edge a weight based on how many scenes the two characters appeared in together. We also used the Louvain algorithm to find eight communities, matched each node to a community, calculated the degree centrality of each character and finally identified the most central characters in each community. I’m going to recreate this igraph object here just for the sake of completion.

library(tidyverse)
library(readr)
library(igraph)

# get s1 edgelist
edgefile_url <- "https://raw.githubusercontent.com/mathbeveridge/gameofthrones/master/data/got-s1-edges.csv"
edgelist <- readr::read_csv(edgefile_url)

# append edgelists for s2-8
for (i in 2:8) {
edgefile_url <- paste0("https://raw.githubusercontent.com/mathbeveridge/gameofthrones/master/data//got-s", i, "-edges.csv")
edges <- readr::read_csv(edgefile_url)
edgelist <- edgelist %>% 
    dplyr::bind_rows(edges)
}

seasons <- 1:8 # <--- edit to restrict to specific seasons

edgelist <- edgelist %>% 
  dplyr::filter(Season %in% seasons)
edgelist_matrix <- as.matrix(edgelist[ ,1:2])

# create igraph network
got_graph <- igraph::graph_from_edgelist(edgelist_matrix, directed = FALSE) %>% 
  igraph::set.edge.attribute("weight", value = edgelist$Weight)

# calculate degree centrality for each node
V(got_graph)$degree <- igraph::degree(got_graph)

# find louvain communities
louvain_partition <- igraph::cluster_louvain(got_graph, weights = E(got_graph)$weight)

#assign communities to graph
got_graph$community <- louvain_partition$membership

# create a separate dataframe of the most central characters in each community - give a custom name to the two small ones
 
community_leads <- data.frame(group = c(), legend_label = c())
for (i in 1:max(got_graph$community)) {
  subgraph <- induced_subgraph(got_graph, v = which(got_graph$community == i))
  degree <-  igraph::degree(subgraph)
  char <- names(which(degree == max(degree)))
  if (grepl("DWARF", char)) {
    char <- "THE FIVE DWARFS"
  } else if (length(char) == 3) {
    char <- "BLACK JACK, KEGS & MULLY"
  }
  new_df <- data.frame(group = i, data_label = char)
  if (nrow(new_df) > 1) {
    new_df <- new_df[1,]
  }
  community_leads <- community_leads %>%
    dplyr::bind_rows(new_df)
}

Now, because we plan to move this network into D3, we will need to recode it in a way that D3 understands, and save it in a data structure Javascript understands. R has good packages for both of these tasks. First, we will use R’s networkD3 package to convert the data structure into a D3-friendly one:

library(networkD3)

# convert got_graph for D3 - this creates a list of two dataframes - one for nodes and one for edges

d3_network <- networkD3::igraph_to_networkD3(got_graph, group = got_graph$community)

# for community coloring in our D3 graph we want our edges to be coded according to their starting node

d3_network$nodes <- d3_network$nodes %>% 
  tibble::rownames_to_column()
d3_network$links <- dplyr::left_join(d3_network$links, 
                                     d3_network$nodes %>% 
                                       dplyr::mutate(rowname = as.numeric(rowname) - 1) %>% 
                                       dplyr::select(rowname, group),
                                     by = c("source" = "rowname")) 
d3_network$nodes <- d3_network$nodes %>% 
  dplyr::select(-rowname)

# capture the degree centrality of the nodes
d3_network$nodes$degree <- V(got_graph)$degree

# capture the weights of the edges
d3_network$links$weight <- E(got_graph)$weight

# capture the community name of each node for our legend
d3_network$nodes <- dplyr::left_join(d3_network$nodes, community_leads)

This gives us two dataframes with all the data we need for transporting into D3. Let’s look at an extract of each:

d3_network$nodes %>% head(3) %>% knitr::kable()
namegroupdegreedata_label
NED2161SANSA
ROBERT2104SANSA
DAENERYS1166DAENERYS
d3_network$links %>% head(3) %>% knitr::kable()
sourcetargetvaluegroupweight
589633192
219623154
2328645121

As a final step we need to save this network as a json file which is the easiest data type to work with in Javascript. We will use the jsonlite package for this:

library(jsonlite)
jsonlite::write_json(d3_network, "json/got_network.json")

Setting up a webpage for our D3 visualization

Now we can move out of R entirely and we have all the data we need in our json file to create our network visualization using D3. First we just need to set up a basic html file that will hold the visualization in a web page. This will need to contain:

  1. An svg (scalable vector graphics) tag that the main D3 visualization will sit in.
  2. Some basic css including text that looks a but like Game of Thrones font, and some styling for our legend which will also come from D3.
  3. Links to the D3 library and to our Javascript file containing our D3 code, which we will put in js/graph_slider.js.
<!DOCTYPE html>
<meta charset="utf-8">
<link href="https://fonts.googleapis.com/css?family=Cinzel+Decorative&display=swap" rel="stylesheet">
<style>
body {
  font-family: 'Cinzel Decorative';
  background-color: #f1e6d2;
}
.links line {
  stroke: #999;
  stroke-opacity: 0.6;
}
.nodes circle {
  stroke: #fff;
  stroke-width: 1.5px;
}
.legend {
  position: absolute;
  right: 1%;
  top: 1%;
}
</style>
<body>
  <div class="row">
    <center>
      <h1>
        Game of Thrones Communities
      </h1>
    </center>
  </div>
<div class="row">
  <center>
    <svg width="1400" height="600"></svg>
  </center>
  <script src="https://d3js.org/d3.v4.min.js"></script>
  <script src="js/graph_slider.js"></script>
  <div id="legend"></div>
</div>
</body>

Coding the D3 visualization

Inside our graph_slider.js file we will write our code for the visualization. If you don’t want to wade through the technical details here, you can see the final graph_slider.js file here.

First, we can already lift some code from here which already does a lot of what we want. If we look through it, there’s one thing we don’t need and a few things we need to add.

We don’t need the different measures of centrality as we are only using degree centrality in this visualization, so we can remove the code associated with var dropdown.

Now we will work on the things we need to add.

Pointing to the data

To point to our json datafile, we need to create a variable with the path to the file and insert this variable into the d3.json function. In my case:

var dataPath = "https://keithmcnulty.github.io/game-of-thrones-network/json/got_network.json"

Coloring the edges

The edges in the code we have lifted are all grey. We want to color them according to the group column of the links dataframe we wrote to our json file. To do this we will need to append a new stroke style property to the links variable which maps the link to its group:

var link = container.append("g")
    .attr("class", "links")
    .selectAll("line")
    .data(graph.links, function(d) { return d.source + ", " + d.target;})
    .enter().append("line")
    .attr('class', 'link')
    .style('stroke', d => color(d.group));  // <- new style

Center the search input

We want to center the search input, which means adding a simple center tag to that variable:

var search = d3.select("body").append('center')  // added a center tag
    .append('form').attr('onsubmit', 'return false;');

Adjusting the slider

We want to center the slider and also ensure that the minimum value is 1, so that we can see the entire network based on a minimum of a single scene together. We will make edits to the slider variable to do this:

  // A slider that removes nodes below the input threshold.
    var slider = d3.select('body').append('p').append('center').text('Minimum number of scenes for connection: ').style('font-size', '60%');  
  // Centered new name and font-size and centered slider
    slider.append('label')
        .attr('for', 'threshold')
        .text('1').style('font-weight', 'bold') // changed minimum
        .style('font-size', '120%'); // bolded and resized 
    slider.append('input')
        .attr('type', 'range')
        .attr('min', 1) // changed minimum
        .attr('max', d3.max(graph.links, function(d) {return d.weight; }) / 2)
        .attr('value', 1) // changed default value to minimum

Create a legend

We want a legend that shows the colors of all the unique communities and names them according to the unique values of the data_label column in the nodes dataframe:

    // add a legend
    var legend = d3.select("#legend")
        .append("svg")
        .attr("class", "legend")
        .attr("width", 180)
        .attr("height", 200)
        .selectAll("g")
        .data(color.domain())
        .enter()
        .append("g")
        .attr("transform", function(d, i) {
        return "translate(0," + i * 20 + ")";
        });
    
    legend.append("rect")
        .attr("width", 18)
        .attr("height", 18)
        .style("fill", color);
    
    // load unique legend names from data_label column
    var legendNames = [];
    var map = new Map();
    for (var item of graph.nodes) {
        if(!map.has(item.group)){
            map.set(item.group, true);    // set any value to Map
            legendNames.push({
                id: item.group,
                groupName: item.data_label
            });
        }
    }
    // append text to legends
    legend.append("text")
        .data(color.domain())
        .attr("x", 24)
        .attr("y", 9)
        .attr("dy", ".35em")
        .text(function(d) {
        return legendNames.find(x => x.id === d).groupName;
        })
        .style('font-size', 10);

Add photos to select nodes

This is probably the hardest task, so I’ve left it for last. Our first step is to take a subset of the nodes which are the six main communities from the legendNames array, and append a slightly smaller circle to those nodes filled with the image of that person. Node that all my image files are named as lower case versions of the node names.

// add photos to all legend names except two
    var photos = legendNames.filter(x => x.groupName !== 'THE FIVE DWARFS' && x.groupName !== 'BLACK JACK, KEGS & MULLY');
    var imgPath = 'https://keithmcnulty.github.io/game-of-thrones-network/img/'
    
   photos.forEach(d => {
        node.filter(x => x.name === d.groupName)
            .append("defs")
            .append("pattern")
            .attr('id', d => 'image-' + d.name)
            .attr('patternUnits', 'userSpaceOnUse')
            .attr('x', d => -degreeSize(d.degree))
            .attr('y', d => -degreeSize(d.degree))
            .attr('height', d => degreeSize(d.degree) * 2)
            .attr('width', d => degreeSize(d.degree) * 2)
            .append("image")
            .attr('height', d => degreeSize(d.degree) * 2)
            .attr('width', d => degreeSize(d.degree) * 2)
            .attr('xlink:href', d => imgPath + d.name.toLowerCase() + '.png');
        
        node.filter(x => x.name === d.groupName)
            .append("circle")
            .attr('r', d => 0.9 * degreeSize(d.degree))
            .attr('fill', d => 'url(#image-' + d.name + ')');
        
            
        })

Now, for this to work, we need to transform the coordinates of each node in the viz, so that any positioning of photos is relative to the specific node that the photo is being placed on. This will require an adjustment to the ticked() function:

function ticked() {
    link
        .attr("x1", function(d) { return d.source.x; })
        .attr("y1", function(d) { return d.source.y; })
        .attr("x2", function(d) { return d.target.x; })
        .attr("y2", function(d) { return d.target.y; });
    node
        .attr("transform", d => "translate(" + d.x + "," + d.y + ")"); // transformed
  }

And we are done with all our customizations. Here’s the final product:

2 thoughts on “Create Amazing Graph Visualizations Using D3.js

  1. Hi!
    Great article, however how can I load the graph on my computer?
    I put the files got_network.json, got.html and graph_slider.js all under the same directory, then I opened got.html on the python web server SimpleHTTPServer (both on Safari and Chrome), in both cases it just shows a cream page with the title Game of Thrones Communities and nothing else.
    Any advice would be greatly appreciated

Leave a Reply

%d bloggers like this: