Transcript Document

Evolution of a Clearinghouse:
From FTP to Map Services
ESRI International Users Conference, August 10th, 2006
Maurie Kelly, Ryan Baxter, David Walrath
Pennsylvania Spatial Data Access
www.pasda.psu.edu
In this presentation….
• Historical background of clearinghouse
•Past and current challenges and initiatives
•Related projects and experience
FTP
&
Metadata
Relational Database
Development
Map Services
&
Customization
Tools
www.pasda.psu.edu
PASDA …
• Is the official public geospatial data clearinghouse for Pennsylvania
PASDA is the Commonwealth’s node on the
NSDI, participates in GOS, the Geography
Network, and the NBII
Serves as a resource
for locating data
throughout PA
Provides widespread
sharing of
Geospatial data
Eliminates the
creation of
redundant data sets
PASDA …
•Established in 1996 to serve PA DEP data (35 data sets) via FTP
• Expanded in 1999 to officially serve as the public geospatial data
clearinghouse for PA
A collaborative effort of:
• PA Geospatial Technologies Office
•Penn State Institutes of the Environment
•State agencies, programs such as PAMAP, and
data partners throughout the region
PASDA…
•Provides access to about 50,000 data files
•Celebrates its 10th anniversary this year and is one of the US’ largest
and most established clearinghouses
• Offices are located on PSU UP Campus
PASDA
PASDA…
• Our services include:
Data Storage
and Access
Users
Data &
Metad
ata
PASDA
Metadata
Outreach
Training & Assistance
Planar Distance Units:meters
Geodetic Model:
Horizontal
Datum
Name:North American
Planar
Distance
Units:meters
Datum ofModel:
1927
Geodetic
Ellipsoid
Name:Clarke
Horizontal
Datum
Name:North American
Planar Distance Units:meters
Datum ofModel:
1927
Geodetic
Ellipsoid
Name:Clarke
Horizontal
Datum
Name:North American
Planar Distance Units:meters
Datum ofModel:
1927
Geodetic
Ellipsoid
Name:Clarke
Horizontal
Datum
Name:North American
Planar
Distance
Units:meters
Datum ofModel:
1927
Geodetic
Ellipsoid
Name:Clarke
Horizontal
Datum Name:North American
Datum of 1927
Ellipsoid Name:Clarke
•PASDA info.
•GIS tutorials
How PASDA Works
Data Partners
-Data Collection
-Mapping
-Documentation
PASDA Staff
-Storage
-Management
-Assistance
Distribution Via the Web
-Retrieval by user
Benefit to Providers:
-Mechanism for data distribution
-Publicity
-Provide service to PA
Benefit to Users:
-Can access & use valuable data
-Avoid duplication of effort
-FREE
PASDA:
Data Cataloging
Description:
Originator:U.S. Environmental
Protection Agency/Office of
Water/OST Basins
Abstract:The U.S. Environmental
Protection Agency's (EPA) Reach Files
are a series of hydrographic databases
of the surface waters of the
continental United States …
Time Period of Content:
Metadata…
Time Period Information:
Single Date/Time:
Calendar Date:1994
Currentness Reference:publication date
Status:
Progress:Complete
Maintenance and Update Frequency:None planned
Spatial Domain:
Bounding Coordinates:
Title:USEPA/OW River Reach
File 3 (RF3) Alpha for CONUS,
Hawaii,
Puerto Rico, and the
Indirect Spatial
U.S. Reference:English-Salmon
Virgin Islands
- New
West Bounding Coordinate:-74.739
East Bounding Coordinate:-73.652
North Bounding Coordinate:45.156
South Bounding Coordinate:44.496
Keywords:
Theme:
Theme Keyword Thesaurus:None
Info about the data
Theme Keyword:RF3 alpha Hydrography River Reach streams stream network rivers
hydrography ArcInfo navigation drainage
Place:
Place Keyword Thesaurus:Geographic Names Information System
Place Keyword:New York NY New York English-Salmon
Access Constraints:none
Use Constraints:none
Calendar Date:1994
Point of Contact:
York
Contact Information:
Contact Organization Primary:
Contact Organization:U.S. Environmental Protection Agency
Contact Person:Dan Parker
Direct Spatial Reference
Method:Vector
Contact Address:
Address Type:mailing and physical address
Address:401 M Street SW (4503F)
City:Washington
State or Province:DC
Postal Code:20460
Country:United States
Contact Voice Telephone:1-800-424-9067
Data Set Credit:McKay, Lucinda; Sue Hanson; Robert Horn; Richard Dulaney; Alan Cahoon; Mark
Olsen; and Thomas Dewald, 1994. The U.S. EPA Reach File Version 3.0 Alpha Release (RF3-Alpha)
Technical Reference. U.S. Environmental Protection Agency, Washington, DC
Spatial Domain:
Bounding Coordinates:
West Bounding Coordinate:-74.739
East Bounding Coordinate:-73.652
Planar Distance Units:meters
Geodetic Model:
Horizontal Datum Name:North
American
Datum of 1927
Ellipsoid Name:Clarke 1866
Types of data on PASDA
•Vector and Raster Data
•DEMs
• DLGs
• DRGs
• DOQQs
• PAMAP
•River Conservation Plans
•Census Data
•Land Use/Land Cover
•Wetlands
•Historic Markers
•Fish Species
PASDA Statistics – End of 2005
• Data Storage
• 6.5 Terabytes of data
• 3 Terabytes in database for interactive
mapping/interactive download
• Access (during last 12 months)
• 11.7 million hits (from 253,000 users)
• 631,000 data downloads requests
• 3.9 Terabytes of data downloaded
• 828,000 online maps generated via web GIS
PASDA Statistics
PASDA Statistics
To whom are we providing assistance?
First Challenge—Expand Services and Capability
beyond FTP
•#1 Challenge for any clearinghouse—Technology
•Technology changes rapidly both for clearinghouse and for users
•Changes in technology mean changes in hardware, software, and architecture of
clearinghouse as well as changes in personnel and training needs.
•Requires regular planning, benchmarking, and vision
Needs Analysis
•1999 PASDA and ATS of Lancaster undertook a needs analysis to define user
requirements for role as official public geospatial data clearinghouse for PA
•User requirements included: customization options, web mapping applications, tutorials
and guides, more data!!
Impact on Clearinghouse
•Increased training needs and knowledge base for staff
•Increased technology, software, and hardware needs
•More users
Results of Needs Analysis Initiative
Strategic Planning effort undertaken and implemented in 2000 facilitated by
Avencia Inc. of Philadelphia.
By 2001, the following were complete
•Development of first relational database using Db2 & SDE
•Pennsylvania Atlas ArcIMS application
•First iteration of Data Wizard customization reprojector/clipping tool
By 2002…
•Statewide DOQQ mosaic in database
•Clipper/reprojector in regular use
•Pennsylvania Atlas ArcIMS application clipping/reprojecting tool
developed
By 2003 Technology Changes on the horizon…..
By 2003 changes in technology were on the horizon…
Mid 2003 User requirements session held. Examined existing tools, data,
and interface. Acquired input on changes and expectations of users—
•Clearinghouse needed integrated web interface
•Clearinghouse needed streamlined metadata
•Clearinghouse needed integrated option for accessing imagery
•Website and html needed to be streamlined
•Search mechanisms needed to be integrated into single environment
•Post user session—staff held strategic planning session and developed
new multiyear plan to implement additional enhancements to
clearinghouse.
Challenge….
How to make all of the options accessible?
Solution 1:
 Approach had been to create new tools for each need
 Approach had been to add links and content to site
Result 1:
Confusing Website
Countless links & access points
Challenge Part 2….
How to make all of the options accessible?
Solution 2:
Our approach this past year
Complete redesign of the clearinghouse…search interface, metadata
architecture, data wizard, integration of applications and map
Services into search interface.
Result 2:
Simplified Website
Single, integrated access point to all data
types & services
Current Architecture…
•“Data cart” option added to Data Wizard
•Users can search for, select, add data to cart, reproject/clip data, and zip all
the data in their cart together into one downloadable file.
•Imagery viewer and download utility created for all imagery in database
Clip &
Reproject
WMS
MapServices
ArcIMS
MapServices
Google
Earth
FTP
.ZIP Files
WebGIS
Applications
NSDI – GOS
Harvesting
Current Architecture…
How did we accomplish this?
One central database table
• One record for each ‘dataset’ in the clearinghouse
• Basic metadata elements (title, keywords, date…)
• Links to FTP files
• Links to SDE database feature classes
• Links to XML metadata files
• Links to WebGIS application URLs
• Links to ArcIMS & WMS MapServices
Current Architecture…
Central table drives the Website and integrates all services
Website
Direct
Download
FTP
.zip
Clip, Reproject
& Download
SDE
Central
Table
Metadata
xml
MapServices
xml
xml
WebGIS
NSDI -- Geospatial One-Stop
ArcIMS -- WMS --Google Earth
Data Access Tools
Search by keyword, county, quadrangle, etc.
Download .zip files
FTP
View aerial photos; zoom to an address
Click on a quarter-quadrangle to download it
DOQQs
View & Interact with data on a map
Clip, reproject & download
PA Atlas
Map & download census 2000 data
All SF1 & SF3 census variables for PA
Census 2000
Live Internet Data
Data Access Wizard
PennCAT - Geography Network
Catalog of MapServices
Unified interface for all data
Add Data to your ‘Data Cart’
Clip, reproject and download
multiple datasets at one time
View XML metadata
Download via FTP
Link to separate
application
Compile datasets for
clipping & reprojecting
Dynamic Content
Specialized Applications
Imagery Viewer and Download Utility
• 4 Terabytes of imagery including aerial photos
DRGs, DOQQs
• 5 unique tiling schemes
• Multiple years of data—spans period from 1993-2005
• All data can be queried via single interface
• Search by address,quadrangle, county
• Select image and provide with download options http ftp mr. Sid or tiff
Specialized Applications:Imagery Viewer
Specialized Applications:Imagery Viewer
Specialized Applications:National Weather Service
Forecast Map Services
Near real time data from NWS including: Apparent Temperature, Dew Point Temperature,Maximum
Temperature, Minimum Temperature, Precipitation Amount, Probability of Precipitation, Relative
Humidity, Sky Cover, Snow Amount, Temperature, Wave Height, Wind Direction, and Wind Speed.
Specialized Applications: Tropical Depression Ivan
• The Ivan viewer was developed to provide information about flood
damage following the storm to the public, emergency managers,
And other stakeholders in the Commonwealth.
• Ivan includes pre and post flood aerial photos, pictures taken at
ground level of flood damage, and supporting framework data.
Ivan Imagery Viewer Interface
City Island after the flood…
Pre-flood imagery of City Island
Flood damage—Photo taken on site and contributed to Ivan viewer
by citizen
Pennsylvania Initiatives that use the clearinghouse…
National Information Infrastructure Initiatives…
Conclusion
•Clearinghouses have evolved from simple FTP
sites to complex information infrastructures
•Clearinghouses serve multiple purposes—access to data
•Access to services, access to applications, metadata
development, and outreach.
•PASDA has undergone regular strategic planning to
develop new initiatives and to benchmark progress.
PASDA has transitioned from a traditional structure to one that
encompasses data, services, and tools for its users.
•Map services and applications are now the hallmark of the
clearinghouse
•Free access to data supports businesses, non-profits, and other
projects and programs throughout the state and nation.
Questions & Comments
ESRI International User Conference, August 10th, 2006
Please contact us at [email protected] if you have
any questions or comments.
More information is available at
http://www.pasda.psu.edu/
Maurie Kelly, Ryan Baxter, David Walrath
Pennsylvania Spatial Data Access
www.pasda.psu.edu