How to Build A Data Visualization Dashboard using D3.JS & JavaScript: Upstream Oil & Gas Data (Part 1 of 2)

In this tutorial I am going to show you how to build a dashboard with JavaScripts and D3.js to visualize upstream oil and gas data. But before that I am going to explain some terminologies, required data, and step by step implementation. This tutorial is going to serve you as a guide on how to build consumable visualizations for clients.

The data and codes can be found on my Github Page.

You can create your own visualizations and share it with audiences here. I am planning to create version 2 of this visualization later with aiming on being interactive and responsive in order to explore data deeper.

Oil & Gas Terminology

In this section I am going to introduce some terminologies. If you are already familiar with following terminologies you can skip to next section.

  • Upstream Oil and Gas: Upstream refers to the location in the supply chain with the proximity to the end users. Oil is produced at well locations hence data related to wells and production are called upstream data.
  • Horizontal & Vertical wells: Traditionally wells drilled vertically from surface to the reservoir rock. Horizontal wells on the other hand are a vertical well with horizontal lateral attached to it. Horizontal wells enable superior well oil and gas production and more contact area with reservoir rock.
  • Well completion: After well is successfully drilled, the process of making well ready for production is called well completion.
  • Shale: Shale is a fine grained sedimentary rock. In petroleum systems, shale is source rock in which oil and gas are formed. Shale contain hydrocarbons but for years considered uneconomical source of energy. However, with advances in drilling and stimulation, shale rocks provide significant source of energy.
  • Hydraulic fracturing: or often called fracing, is the process of stimulating the unproductive reservoir rocks by pumping large quantities of water and propping agent (sand or ceramic) to create fractures within reservoir rocks. Fractures enable uneconomical wells to produce at economic rate as they facilitate fluid flow inside rocks. Fractures are often viewed as highways which oil and gas can use to travel to the wellbore as opposed to using jam packed traffic inside the cities.

All wells in the database are hydraulically fractures horizontal wells.

  • Important measurable design parameters of hydraulic fracturing are listed below:
  • Number of stages: In order to effectively stimulate wells, the length of well is divided into N number of stages, and each stage isolated and stimulated separately.
  • Frac Fluid: Fracturing fluid is mainly consists of water and gelling agent to increase viscosity of water. The amount of pumped fracturing fluid is important design parameter.
  • Propping agent, and total proppant mass: In order to keep fracture opens, some materials need to be pumped to ensure it. It is very important to keep fractures open as they ensure maximum oil and gas production rate from fractures. Traditionally, Frac sands (mined sand) is used as propping agent. Another alternative to frac sands is proppant agent made of ceramics. The type and amount of propping agents are important design parameter of farcturing process.
Illustration of hydraulic fracturing process [link to image]

Data

There are two main components to building dashboard to be consumed for Upstream oil and gas data:

  • Bivariate plots comparing different data attributes, and/or different histograms that provide insights about the data
  • GIS analysis on well locations, operating companies locations etc.

Therefore, two types of data is needed to acomplish this task:

  • Database of wells containing location, inheritance information, and other important info such as production completion etc.
  • Shapefile of geographical areas that wells are located in

Data required for this analysis is located in GITHUB project file:

  • EF_Counties.geojson contains shapefile of counties that wells are located in. This case study is focused on Eagle Ford Shale located in Texas. Therefore, shapefile of Texas is trimmed to counties containing wells.
  • EfData.csv contains CSV data of wells. Each row represent a well that contains multiple data attributes including coordinates that help us display wells on the map. Description of the data is given in the github readme file.

This data is scraped from The Railroad Commission of Texas. I scraped the data and cleaned it up.You can use it freely. However, in return you can share with me any improvement that you have done on my version. Remember, sharing is caring!

EfData.csv contains 21 columns and 23,371 wells (#rows)

Dashboard Structure

The dashboard structure that I chose to visualize this dataset is a Quad Plot contain four different visualizations as plotted below.

  • Panel 1 contains number of wells in top counties. Since wells are scatter over entire region, here I limited my bar plot to counties with at least 700 wells.
  • Panel 2 is the map of Texas counties that contain Eagle Ford wells color coded by the producing fluid. Since wells drilled in Eagle Ford Shale produce different fluid (oil, gas condensate, etc.) it is important to distinguish wells for each fluid category.
  • Panel 3 contains the average cumulative production of wells drilled each year.
  • Panel 4 demonstrates best operator year by year as identified by operator with highest average production and with at least 20 wells in the database (in order to be statistically significant).
Upstream dashboard containing 4 panels

Dependencies

I am going to use D3.js v4 for this case study as I created this dashboard while ago and didn’t upgrade to v5. Pleases feel free to modify this dashboard.

Reading the Data

The first task is to read data. We need to read csv and geojson data. d3.csv and d3.json commands can accomplish these task as seen in the following code. Two nested loops will, read EfData.csv and Ef_Counties.geojson into EfData and json variables.

setTimeout(function() {var formatComma = d3.format(“,”)
var margin = {
top: 15,
right: 100,
bottom: 15,
left: 100
};
// — — load the data
d3.csv(“./data/EfData.csv”, function(EfData) {
d3.json(“./data/EF_Counties.geojson”, function(json) {
// ------ Code Blocks Here
}
}
},100)

In the next article I am going to show you how to build this database step by step.

Please read the part 2 of this article here (Link).

Thanks for reading! My name is Amir Nejad,PhD. I’m a data scientist and editor of QuantJam , and I love share my ideas and to collaborate with other fellow data scientists. You can connect with me on Github, Twitter, and LinkedIn.

QuantJam:

You can see my other writings at:

PhD. Engineer | Data Scientist | Problem Solver | Solution Oriented (twitter: @Dr_Nejad)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store