Current Perspectives on Transportation Metadata Needs

2006 TRB Annual Meeting Frances D. Harrison Chair, TRB Data and Information Technology Section Subcommittee on Metadata January 22, 2006 Spy Pond Partners

Agenda

• Metadata – what and why • • • • • • • Metadata in transportation Library Geospatial ITS – Archived Traffic Data ITS – Road Weather Data TransXML – road and bridge design, construction, safety TRB Data Section Subcommittee on Metadata • • Mission and Role Activities and Next Steps 2

Don’t you hate it when…

• You ask five people in your agency the number of highway system miles and get five different answers • • You spend an hour a day answering the same questions about your agency’s data Your “data guru” quits and leaves gigabytes of partially cleaned data with cryptic documentation • You spend thousands of dollars building a data set only to discover that it already existed in another agency • • You try to combine data gathered from five different corridor studies sponsored by different agencies but can’t fit the pieces together You can’t use half of the data you find because you don’t know what it means or how good it is 3

Wouldn’t it be great if…

• Agencies had policies in place to require good, complete documentation of all data sets – and devoted the resources to make this happen • There were standard templates for documenting every major kind of transportation data • There were tools to produce and validate this documentation • There was an easy way for agencies to share their data and move towards standardized data definitions and coding schemes • There was agreement about how to measure and describe data quality for different types of data • There were better and quicker ways of finding data on the web than Google 4

The Solution: Metadata!

Standards Policies Tools Repositories Workflow Training 5

Definition of Metadata

“Information about a thing, apart from the thing itself”

Thing

Book -Ned Batchelder “Structured data about an object that supports functions associated with the designated object” -J. Greenburg Photograph Web Page GIS Coverage Data Element

Metadata

Title Author Publication Date ISBN Number Subject Photographer Date URI Title Language Location Accuracy Format Element Name Data Type Length Possible Values 6

Card Catalogue

7

Document Properties

8

Geospatial Metadata

9

Data Dictionary – Attribute Information

Source: Traffic Management Data Dictionary 10

Metadata for Sharing Photographs – Flickr Folksonomies

11

Metadata Maximizes and Preserves Data Value

Discovering Understanding Sharing 12

But Metadata is Hard

1.

2.

3.

4.

5.

6.

7.

Too abstract Too complicated Not sexy Not easy Tedious Time consuming Expensive 8.

9.

Hard to get agreement Not on the critical path 10.

Not enforced 11.

Low priority 12.

Stepchild of the stepchild 13

Do we need a 12 step program for metadata?

1.

2.

3.

4.

5.

6.

7.

Recognize there is a problem Believe there is a way to solve it Commit to doing something about it Examine your current practices Publicly admit that things need to change Prepare to change the status quo Get help from colleagues and other experts 8.

9.

Identify specific practices to be improved Correct them 10.

Institutionalize the process in your agency 11.

Periodically reaffirm why you are doing this 12.

Help others by sharing your successes and lessons learned 14

Metadata in Transportation - examples

• Library • Geo-spatial • ITS – Archived Traffic Data • ITS – Road Weather Data • TransXML - data exchange formats for road and bridge design, construction, safety 15

Existing Standards (to name a few…)

• • • Dublin Core – cross-domain resource description MODS, MARC, METS –metadata standards for digital libraries FGDC CSDGM – Metadata for geospatial datasets • • • • • • ISO 19115/19139 Geographic Information - Metadata ISO 11179 – Metadata Registries IEEE 1489 – ITS Data Dictionaries ISO 14817 – Meta Attributes for ITS/TICS Data Dictionaries IEEE 1487 – Metadata Registries for ITS (based on ISO 11179 and IEEE 1489) DDI – Data Documentation Initiative (social science survey data documentation) 16

National Transportation Library – Modified Dublin Core

17

Transportation Research Thesaurus

18

FGDC Metadata

• • Executive Order 12906 (1994) • • Mandated creation of metadata standards in conjunction with the National Spatial Data Infrastructure (NSDI) Federal Geographic Data Committee (FGDC) – interagency group charged with creation and maintenance of the standards Widely adopted and used • • Wealth of resources: tools, tutorials, training Enabler of metadata clearinghouses – access to wide variety of geospatial datasets • Harmonization with ISO 19115 19

FGDC Content Standard for Digital Geospatial Metadata

20

Archived ITS Data

• • ASTM E2259-03 Standard Guide for Archiving and Retrieving ITS-Generated Data • • Published June 2003 General description of metadata types: archive structure, processing documentation and data collection system ASTM Task Group E17.54.02.1: “Standard Specification for Metadata Content for ITS-Generated Data • • Based on FGDC – given wide acceptance and tool availability Tackling specific issues related to traffic data – e.g. quality, aggregation methods 21

Clarus

• National repository of road weather data – network of networks • • Separation of Producers and Consumers - Metadata Essential • • 80 elements defined Based on quality checking and user requirements Data exchange formats • Need to address data sharing between “Weather World” and “ITS World” 22

TransXML

• Transportation Data Exchange Schema – NCHRP 20-64 • • Four areas • • • • Roadway Design (integration of GIS data, construction cost estimates) Construction (bidding, payments, status tracking) Safety (crash data and highway safety analysis) Bridge Design (verification of structural analysis) Definition of data structures – entities, attributes and relationships (UML) • XML Schema posted for comment: www.transxml.org

• • • Sample applications end of January Final Report – June 2006 Presentation at AASHTO IS (June 4-7 – Ft Worth, TX) 23

Example: Data Exchanges for Construction Progress

Field Data Collection Systems Laboratory Systems Installed Quantities Pay Item Quantities Materials Material Test Results Placement Location Material Samples Tests Performed Test Outcomes Project Engineer Progress Reporting Progress Payment System

24

25

TransXML – Issues

• • • • • • • • Leverages existing schema and standards – different styles, overlapping elements, moving targets GML LandXML Global Justice NHTSA Crash Records (MMUCC and FARS) AASHTOWare data schema aecXML Balance between consistent framework and niche schemas • Stakeholder participation lower than anticipated • Never enough resources for communication, education and liaison • Chicken and egg considerations • • “Vendors will only act if agencies demand it” “Agencies will come on board if the vendors support it” 26

Conclusions

• • Needs and technology converging to enable advances in practice Good models exist – foundation for continued work • • More standards and models needed • • • • Dublin Core or FGDC extensions and refinements for different data objects Data dictionaries and schemas for common data exchanges Taxonomies, controlled vocabularies Code lists and format rules for shared attributes Standards are not enough - tools, training, policies, practices • • Great need for coordination and information sharing across diverse efforts Crosswalks – to promote understanding and enable harmonization 27

TRB Metadata Metamorphosis

• • • 1998-2005 Metadata Subcommittee operated under Urban Data – led by Marcus Wigan • • • Research Education Coordination Need for broader approach • • • Metadata is a cross-cutting issue – not appropriate as a subcommittee under Urban Data Metadata is key to data sharing – but requires leadership and coordination Metadata is abstract, not widely understood hard to get people’s attention • How to get to the next level? Working Group – February 2005 • • • Joanna Zmud chair Mission: Recommend plan of action for the Data and Technology Section by summer meeting in August Outcome: Section subcommittee immediately – morph into Task Force 28

TRB Metadata: The Next Generation

• Section Subcommittee – August 2005 • Members from most Data Section Committees + Library Science • Mission and Roles • • • • • • • Promote common understanding and appreciation of metadata value Share information about metadata related activities across transportation data communities Identify priority needs for new metadata standards Promote research that provides technical groundwork for standards Promote coordination and consistency across standards efforts Promote best practices in use of metadata Promote metadata training and tools development 29

TRB Metadata Subcommittee – Activities To Date

• Monthly Conference Calls • • • Compiling Resource Lists: Standards, Tools, Implementation Examples • • • • • • • • Draft Research Agenda – Eight Initiatives White Paper State of the Practice Survey – help to set priorities Best Practice Synthesis End User Needs Investigation MPO Use of Metadata for Modeling and GIS-T Gap Closure Feasibility Study for Metadata Registry or Clearinghouse Crosswalk and Code List Catalogue Comments and Input Welcome – [email protected]

30

Future Task Force Product: Strategic Plan for Metadata

• Establish need for and benefit of metadata standards • Recommend and prioritize types of standards for development • Identify potential barriers to acceptance • Identify coordination needs across standards efforts • Identify stakeholders and participants • Recommend methods to promote use 31

Acknowledgements

• • • • • • • • • • • • • Kuo-Ann Chiao (NYMTC) Scott Dahl (US Census Bureau) Ralph Gillmann, FHWA James Hall (University of Illinois, Springfield) Joyce Koeneman (RITA-NTL) Guy Rousseau (ARC) Jack Stickel (Alaska DOT) Xiaduan Sun (University of Louisiana) Shawn Turner (Texas A&M) Anita Vandervalk (CSI) Marcus Wigan, Oxford Systematics Eric Ziering (CSI) Joanna Zmud, NuStats 32