Mash-up of Linked Government Data from http://data.gov Li Ding, Jim Hendler and Deborah L.

Transcript Mash-up of Linked Government Data from http://data.gov Li Ding, Jim Hendler and Deborah L.

Mash-up of Linked Government Data from http://data.gov

Li Ding, Jim Hendler and Deborah L. McGuinness Tetherless World Constellation, Rensselaer Polytechnic Institute June 24, 2010

Raw Government Data Now

2009

“Openness will strengthen our democracy and promote efficiency and effectiveness in Government.” --- President Obama data.gov online Putting Government Data online “Open Government Directive” released data.gov relaunch with semantic web featured data.gov.uk online

2010 …

Semantic Web featured at data.gov

3 http://www.data.gov/semantic/ http://www.data.gov/semantic/data/alpha • leveraged contributions from the Tetherless World Constellation at RPI • published 6.4 billions of triples (almost doubled LOD cloud – 13 billion triple in total) • hosted triple store (virtuoso) and open source RDF mashups

The Data-gov Wiki: Innovations at RPI

4 The Data-gov Wiki explores and educates the use of semantic web technologies, esp. linked data, in producing, processing and utilizing government data from data.gov.

Demo Data Tutorial Video The Data-gov Wiki is run by the Tetherless World Constellation at RPI, headed by Professor Jim Hendler and Deborah McGuinness and led by Li Ding. Other student team members include: Dominic DiFranzo, Sarah Magidson ,James Michaelis, Alvaro Graves, Adam Bell, Jin Guang Zheng, Xian Li, Tim Lebo, Gregory Todd Williams, Peter Coons, Zhenning Shangguan, Devin Gaffney, William Cooper, Brian Zaik, and Johanna Flores .

The Data-gov Wiki - Architecture

5 Data Web

Usage

LGD in RDF Linked Data …

Conversion Enhancement

LGD: Linked government data

Open Data => Visualization => More

6 • Open Data: available for public use • Visualization: easy to understand • Mashups: make it more meaningful • Provenance: make it accountable Table (raw data) Map (books per state) Map (books per capita per state) Created by Xian Li, PhD student at RPI, http://data-gov.tw.rpi.edu/wiki/Demo:_Library_Books_Per_Capita,_by_State Source: Dataset 353 (State Library Agency Survey: Fiscal Year 2006, Institute of Museum and Library Services)

Data.gov

CASTNET Ozone (CSV)

Example Mashup

epa.gov

CASTNET Site (CSV) 1 Convert raw dataset into linkable RDF 2 query multiple RDF dataset via SPARQL end point 4 surf to EPA applications

3 drill down for details Exhibit Visualization API Data Mashup Visualization Mashup Web Application Mashup

Created by Dominic DiFranzo, PhD student at RPI, http://www.data.gov/semantic/Castnet/html/exhibit

Mashup US UK Foreign aid

• Sources – http://data.gov

– http://data.gov.uk

• Discovery and Explanation Pakistan India AID US >UK UK > US Major aids from US Economic/Security Assistance, Development Assistance, … Child Survival and Health, … Major aids from UK Health, Gov and Civil Society,..

Health, Economic, … 8 Created by James Michaelis, PhD student at RPI, http://data-gov.tw.rpi.edu/demo/linked/aidviz-1554-10030.html

Adding Social Factor to Mashups

9 • Import socially contributed data, e.g. DBpedia • Let users contribute – links – feedbacks Raw Data Publish* Enhance* RDF consume* feedback User Other Social Web Apps

Social Mashup: Gov Data + DBpedia

10 Category:Wildfires In The United States Budget on wildfire “DOI” and “USDA” (OMB) Wildland fire (NIFC) Created by Li Ding, researcher at RPI, http://data-gov.tw.rpi.edu/demo/stable/demo-1187-40x-wildfire-budget.html

Social Mashup: Web 3.0 Linking

Data-gov Wiki

“POTUS” dbpedia:Barack_Obama

whitehouse DBpedia Wiki Text

*[[skos:altLabel::POTUS]] *[[foaf:firstName::Barack]] [[foaf:lastName::Obama]] *[[owl:sameAs::http://dbpedia.org/resource/Barack_Obama]] *[[owl:sameAs::http://rdf.freebase.com/ns/en.barack_obama]] *[[owl:sameAs::http://data.nytimes.com/47452218948077706853]]

RDF data DBpedia

(a) White house visitor search for President Obama

White House Visitor Record

(b) Web 3.0 site linking white house record to dbpedia Created by Dominic DiFranzo, http://data-gov.tw.rpi.edu/demo/stable/white-house-visitor/top100-visitees.php

Social Mashup: User Feedback

• Mashup multiple time series • Support users to feedback (contributing News) 12 Created by Sarah Magidson, http://data-gov.tw.rpi.edu/demo/linked/demo-401-usps-news.html

More Mashups: Using Web Tools

13 SPARQL results (XML) can be converted into other formats (e.g. JSON, CSV) as input of other Web tools: Yahoo Pipes, IBM Many Eyes, Microsoft Web n gram Service, …

More Mashups: Provenance

14 • Critical to accountability • Demo => Dataset => Agency – Where data come from?

• Agency =>Dataset => Comments – Support users’ feedback

Agency Dataset Demo

Even More …

• Applications – More raw data, data catalog, links, hub datasets – More tools, esp. visualization Web APIs – Friendly UI • Research – Data Integration: smart and scalable – Data access: search, social interaction,… – Provenance: source, versions, changes,… – Reliability: trust, persistency, quality… 15

Conclusion

• 6.4 billions of triples from data.gov

• “data + visualization + mashup” is powerful • Low-cost prototypes, not difficult, undergraduates and Webmasters can do it • Open source tools, data, demos and tutorials are available for education