Transcript Mapping Brown Bag

Transforming legacy data into state-ofthe-art interactive visualizations
September 6, 2013 | Washington , D.C.
Mark Elbert
Director, Office of Web Management
U.S. Energy Information Administration
Independent Statistics & Analysis
www.eia.gov
What to do when your data is in legacy table
structures?
Mark Elbert, Washington, D.C.,
September 6, 2013
2
Challenge: transform EIA’s data publishing
• chose electricity data, with over 400,000 time series, to break
the paper-centric publishing model
• different customers consume data in different ways:
– researchers and energy analysts
– classroom of students
– Hill staffers
– application developers
• push the boundaries of interactive charting and mapping on
the web
Mark Elbert, Washington, D.C.,
September 6, 2013
3
The Internet’s fallow period: ’03 - ’10
• The internet turned into a powerful computing platform
despite Microsoft
– Telcos gave us multi megabit bandwidth
– Google’s Chrome started the new browser war for speed
– processors continued to improve
• open-source JavaScript libraries matured
– jQuery / jQueryUI
– jVectorMaps
– Highcharts
Mark Elbert, Washington, D.C.,
September 6, 2013
4
Web Data Ingestion and Application
Architecture
legacy
tables
Data
mapper
Filtering &
collating
Indexing
engine
Data writer
Web DB
(JSON)
Mark Elbert, Washington, D.C.,
September 6, 2013
5
The Electricity Data Browser
• aggregate data
• drill-downs to individual
plant data
• user generated
visualizations:
– Line charts
– Column charts
– Animated maps
• links to analysis, data API
• Cross links to U.S. Energy
Mapping System
Mark Elbert, Washington, D.C.,
September 6, 2013
6
Data ecosystem
{"request":{"command":"series","series_id":"ELEC.PLANT.GEN.6
417-WATALL.A"},"series":[{"series_id":"ELEC.PLANT.GEN.6417-WATALL.A","data":[["2012","244685"],["2011","390119"],["2010","2510
53"],["2009","377491"],["2008","181726"],["2007","102447"],["200
6","292362"],["2005","263951"],["2004","471543"],["2003","44429
3"],["2002","422206"],["2001","285082"]],"name":"Net Generation
: Center Hill : Conventional Hydroelectric : All Primemovers :
Annual","units":"megawatthours","f":"A","unitsshort":"","descriptio
n":"","copyright":"None","source":"EIA, U.S. Energy Information
Administration","iso3166":"USA-TN","lat":"36.0972","lon":"85.8274","start":"2001","end":"2012","updated":"22-MAR-13
03.12.35 PM"}]}
Mark Elbert, Washington, D.C.,
September 6, 2013
7
Demo:
Mark Elbert, Washington, D.C.,
September 6, 2013
www.eia.gov
8
Mark Elbert, Washington, D.C.,
September 6, 2013
9
Mark Elbert, Washington, D.C.,
September 6, 2013
10
Mark Elbert, Washington, D.C.,
September 6, 2013
11
Mark Elbert, Washington, D.C.,
September 6, 2013
12
Mark Elbert, Washington, D.C.,
September 6, 2013
13
Mark Elbert, Washington, D.C.,
September 6, 2013
14
each data set is a complete n-Cube
Complete n-Cube
Ragged n-Cube
Mark Elbert, Washington, D.C.,
September 6, 2013
15
Open Source libraries = 2/3 code
Name
jQuery
jQuery UI
Highstocks
Description
helper library; simplifies
JavaScript coding
user interface control
charting library
License
open-source
Version Size
v1.7.1
92kB
First published Dependencies
Jan-06
-
open-source
v1.8.7
commercial ($80) v1.1.4
202kB
117kB
Sep-07 jQuery
Oct-11 jQuery
jVector Map
map infographics lib
USA map definition USA map definition
open-source
open-source
v0.1
v0.1
23kB
47kB
Feb-12 jQuery
Feb-12 jVector Map, jQuery
SlickGrid
JQuery Event Drag
fast, scalable grid component
extends jQuery; used by Slick
open-source
open-source
v2.0
v2.0
12kB
5.1kB
Mar-09 jQuery, jQuery.event. drag
jQuery
jQuery mousewheel
simplifies mousewheel
programming
creates wait screen effects
help tips
loads library as needed
objectoriented extension for
JavaScript
pattern matching framework
url hash management
logic-less template system
creates modal panel (greys rest
of screen)
provides JSON support for
obsolete browsers
model-view-controller
framework
event framework
cross-browser localStorage API
open-source
v3.0.6
3.5kB
Apr-07 jQuery
open-source
open-source
open-source
open-source
v0.4
v1.0
v0.6.1
v1.4
4.1kB
22kB
14.2kB
4.99kB
Jun-09 jQuery
Apr-10 jQuery
Jan-11
Apr-10 jQuery
open-source
open-source
open-source
open-source
v0.7.1
v1.1
v0.5
v1.4.2
5kB
2.8kB
14kB
23kB
Apr-11
Aug-11
Oct-09
Oct-07 jQuery
-
open-source
v1.0
5kB
Nov-10
-
open-source
v1.1
7kB
Nov-10
-
open-source
open-source
v0.7.4
v1.0
3.1kB
4.6kB
Nov-10
Feb-11
-
Loadmask
qTips
Curl
Classy
Crossroads
Hasher
Mustache
SimpleModal
JSON2
PureMVC
js-Signals
Kizzy
Mark Elbert, Washington, D.C.,
September 6, 2013
-
16
Questions?
Mark Elbert
[email protected]
[email protected]
Mark Elbert, Washington, D.C.,
September 6, 2013
17