HTML, JavaScript, SVG, D3: a foot in the door

Dr. Paul Harrison

Presenting data

Say we want to present our data, or a complex idea, using interactive software.


We might like it to:

  • be easy for many people to access.
  • be relatively easy to develop.
  • keep working in future decades.

Writing a web page scores highly on all of these.

The big picture

We will be using:
HTML
to create a web-page.
<!doctype html>
SVG
to create pictures in the web-page.
<svg id="plot">
</svg>
JavaScript
to run code in the web-page.
<script>
D3
a module that helps turn data into pictures.
d3.select("#plot")


You should also go on to learn:
CSS
to format and lay out your web-page.
<link rel="stylesheet" href="style.css">

Ok, let’s go make a web page, then review what we did.

HTML

We’ve seen we can write HTML using a text editor.

The HTML document is structured as a tree of tags within tags within tags.

Once we start using JavaScript, we can access the tags in the document as elements in the “Document Object Model” (DOM).

SVG

Scalable Vector Graphics (SVG) is an XML-based format for vector graphics.

SVG can be a stand-alone file or embedded in an HTML file.

SVG in a web page is part of the DOM, and we can modify is using JavaScript.

JavaScript

JavaScript allows you to include code in your web page.

Since it runs in a web page we can do some unusual things:

  • Respond to events.
  • Update the DOM.
  • Fetch data from the web.


Note: JavaScript is completely unrelated to Java.

Modules

Old way: A script file that defines a global variable.

New way: JavaScript now has is a proper module system, ESM, and import statements.

BUT you need a web server to use ESM modules.


So:

  • JavaScript packages will often explain how to use their package the new way.
  • At least initially, we’ll use the old way.

D3

D3 is a module that provides a useful toolkit for presenting data interactively.

It doesn’t give you a function to make a plot,
it gives you the tools you need to make it yourself.

  • Simplified DOM manipulation.
  • “Joining” data with DOM elements.
  • Animation.
  • Scale calculations such as position and color.
  • Fetching data.
  • Helpers for certain interactions such as drag-and-drop.
  • …

Data in JavaScript

Numbers: 123, 3.14, 6.02e23

Strings: "hello"

Arrays: [0, 1, 2, "hello"]

Objects: {x: 5, y: 10}


By nesting data in arrays and objects, you can build complex data structures.

This was so useful it became a standard data format called JSON.


Arrays and objects in JavaScript are similar to lists and dicts in Python.

Example dataset

Spatial RNA-Seq was performed on a section of mouse brain.

Following the Seurat analysis, I wrote out some of the results with this R code:

library(jsonlite)
json_script <- paste0("let spots = ", toJSON(dataset, pretty=TRUE), ";")
writeLines(json_script, "spots.js")

Examine the file spots.js.


(I’ve written the data out this way so we can work without a web-server. You can read CSV and JSON files directly if they are on a web-server.)

Ok, let’s make an interactive plot.

Comments

JavaScript got better

JavaScript has a lot of history, including some mis-steps. Since we can’t just go and break millions of websites, those mis-steps can’t be erased.

However we can use new features that avoid those mis-steps:

  • Put "use strict"; at the top of scripts to enable strict mode.
  • == has complex rules for equality, so we now prefer ===.
  • Variable declarations with var had some subtle pitfalls, so now we use let.
  • Writing modular code relied on an ad-hoc module system, but now there is a formal module system.

Used tastefully, JavaScript is now quite the nice language.

A data scientist dabbler’s perspective

There is a vast and ever changing range of modules, frameworks, and tooling for JavaScript, aimed at a variety of users with differing needs.

You are allowed to ignore all this.


Packages like jQuery and D3 provide convenient wrappers around built-in features such as document.querySelectorAll(). They may save some typing, but it’s also fairly easy manipulate the DOM by yourself.

  • Today I use D3 mostly for its ability to synchronize (“join”) datasets with DOM objects.

Other packages and tooling are aimed at larger projects, such as React.

  • Today, instead of using React/etc, we wrote one big idempotent function called update(), and called it whenever needed.

Idempotent

An idempotent operation has the same result no matter how many times you run it.


For example, our update() function updates the DOM to be a function of our internal state.


The same code sets up the initial plot and, when the state changes, is used to update the highlighted region.



As a project grows we might optimize the code, but it should always act as though it was written in this simple way. D3’s .join() is step down this path. Rather than deleting everything and starting over each time, .join() creates or destroys DOM objects only as needed.

Further things

D3 Scales

My example data already had a convenient range of values.

Usually you will need D3 scales mapping the data values to x and y positions in your SVG!

Scales have an inverse mapping, so you can work out where the mouse is pointing in the original data space.

  • Color scales are available.

  • D3 can also help draw a scale axis.

let myScale = d3.scaleLinear([0,1], [50,450]);
myScale(0.5);        // 250
myScale.invert(250); // 0.5

d3.select("#example")
  .append("g")
  .attr("transform", "translate(0,10)")
  .call(d3.axisBottom(myScale));

D3 animation

D3 supports animation.


Insert

    .transition()

into your D3 code, and subsequent updates to attributes will be animated.

CSS and styles

Styles can be applied to HTML tags:

<p style="color: #ff8800">Hello</p>

Hello


Styles can be applied to the whole document using CSS.

<style>
p {
    color: #ff8800;
}
</style>


The CSS can be put in an external file.

<link href="style.css" rel="stylesheet" />

CSS and styles

I especially recommend you learn to use the modern grid layout system.

<div style="display: grid; grid-template-columns: 1fr auto;">
  <div>Hello</div> 
  <div>to</div> 
  <div>grid</div>
  <div>layout!</div>
</div>

Hello
to
grid
layout!

Run a local web-server

Running a web-server lets you:

  • Use modern ESM modules.
  • Access data files without the hack of turning them into JavaScript scripts.

There are many ways to run a local web server.


Install python and use:

python3 -m http.server -b 127.0.0.1


Install node and use:

npx http-server -a 127.0.0.1


In both cases, I specified to listen to the loopback IP address 127.0.0.1 which is accessable from your own computer but not the local network.

Integrate with R or Python

R can include JavaScript widgets, for example in Quarto documents or Shiny apps.

Jupyter notebooks can also output HTML and JavaScript.

The final step of this workshop is to put your page online using GitHub Pages.