Transcript Implementing Geographic Information System Grid Services Using Distributed Messaging Systems Marlon Pierce Community Grids Lab Indiana University December 15, 2004

Implementing Geographic
Information System Grid Services
Using Distributed Messaging
Systems
Marlon Pierce
Community Grids Lab
Indiana University
December 15, 2004
1
SERVOGrid Basics


SERVOGrid is our project to build a distributed computing
infrastructure to support earthquake simulation codes.
Distributed Computing Infrastructure
• Services for remotely executing applications, managing distributed files in
logical units, monitoring applications, archiving user projects, and so on.
• Variously called “Grids” (DOE), “Cyberinfrastructure” (NSF), “eScience” (UK e-Science program), ….

The above is sometimes referred to as an “Execution Grid”.
• We designed this around a web service architecture.
• See links at the end for more information.

But what about the data?
• Need ways to access remote data sources through programming interfaces.
• Practice often is to download (via FTP) entire catalogs and hand edit to
get data you want in the format you want.
• We need to automate this.
2
SERVO Apps and Their Data

Disloc: handles multiple arbitrarily dipping dislocations (faults)
in an elastic half-space.
• Relies upon geometric fault models.

GeoFEST: Three-dimensional viscoelastic finite element model
for calculating nodal displacements and tractions. Allows for
realistic fault geometry and characteristics, material properties,
and body forces.
• Relies upon fault models with geometric and material properties.

Virtual California: Program to simulate interactions between
vertical strike-slip faults using an elastic layer over a viscoelastic
half-space.
• Relies upon fault and fault friction models.

RDAHMM: Time series analysis program based on Hidden
Markov Modeling. Produces feature vectors and probabilities
for transitioning from one class to another.
• Used to analyze GPS and seismic catalogs.
3
Where Is the Data?

QuakeTables Fault Database
• SERVO’s fault repository for California.
• Compatible with GeoFEST, Disloc, and VirtualCalifornia
• http://infogroup.usc.edu:8080/public.html

GPS Data sources and formats (RDAHMM and others).
• JPL: ftp://sideshow.jpl.nasa.gov/pub/mbh
• SOPAC: ftp://garner.ucsd.edu/pub/timeseries
• USGS: http://pasadena.wr.usgs.gov/scign/Analysis/plotdata/

Seismic Event Data (RDAHMM and others)
• SCSN: http://www.scec.org/ftp/catalogs/SCSN
• SCEDC: http://www.scecd.scec.org/ftp/catalogs/SCEC_DC
• Dinger-Shearer: http://www.scecdc.org/ftp/catalogs/dinger-shearer/dingershearer.catalog
• Haukkson: http://www.scecdc.scec.org/ftp/catalogs/hauksson/Socal
4
What Are Web Services?
Web Services framework is a way for
doing distributed computing with
XML.
•
•
•



XML provides cross-language support
Suitable for both human and
application clients.
We built SERVOGrid’s execution grid
services this way.
We are building data grid components
in the same fashion.
•
Allows us to build general purpose
client environments like Web portals
and command line tools.
Browser
Appl
Web
Server
WSDL
SOAP
WSDL
SOAP
Web
Server
WSDL

WSDL: Defines interfaces to functions
of remote components. This is the
programming interface.
SOAP: Defines the message format that
you exchange between components.
This carries the messages over the
network.
Web Services are not web pages, CGI,
servlets, applets.
WSDL

JDBC
DB
5
Browser Interface
HTTP(S)
User Interface Server
WSDL WSDL WSDL WSDL
SOAP
WSDL
DB Service 1
SOAP
WSDL WSDL
Job Sub/Mon
And File
Services
WSDL
Viz Service
JDBC
DB
Host 1
Operating and
Queuing
Systems
Host 2
IDL
GMT
Host 3
6
GIS Standards for Data Grids

Geographic Information Systems
• ESRI: commercial company with many popular GIS
products.
• Open Geospatial Consortium (formerly OpenGIS
Consortium).
• We will focus on OGC since they define open and
interoperable standards.

What are the characteristics of a GIS system?
• Need data models to represent information
• Need services for remotely accessing data.
• Need metadata for determining what is stored in the services.
7
Open GIS Services


GML abstract data models can encode data but you need
services to interact with the remote data.
Some example OGC services include
• Web Feature Service: for retrieving GML encode features, like faults,
roads, county boundaries, GPS station locations,….
• Web Map Service: for creating maps out of Web Features
• Sensor Grid Services: for working with streaming, time-stamped data.

Problems with OGC services
• Not (yet) Web Service compliant
 “Pre” web service, no SOAP or WSDL
 Use instead HTTP GET/POST conventions.
• Often define general Web Service services as specialized standards
 Information services
 Notification services in sensor grids
8
GML: A Data Model For GIS



GML 3.x is a interconnected suite of over 20 connected
XML schemas.
GML is an abstract model for geography.
With GML, you can encode
• Features: abstract representations of map entities.
• Geometry: encode abstractly how to represent a feature
pictorially.
• Coordinate reference systems
• Topology
• Time, units of measure
• Observation data.
9
Example Use of GML


The SCIGN (Southern
California Integrated GPS
Network) maintains online
catalogs of GPS stations.
Collective data for each site is
made available through
online catalogs.
• Using various text formats.



This is not suitable for
processing, but GML is.
GML can be used to describe
GPS using Feature.xsd
schema, with values encoded
at GPS observations.
www.crisisgrid.org.
10
Anatomy of WFS (G. Aydin)

WFS provides three major services as described in OGC specification:
• GetCapabilities: The clients (WMS servers or users) starts with requesting a
document from WFS which describes it’s abilities. When a getCapabilities request
arrives, the server dynamically creates a capabilities document and returns this.
• This is OGC’s formalization of metadata, so important to GEON, ESG, etc.
• DescribeFeatureType: After the client receives the capabilities document he/she
can request a more detailed description for any of the features listed in the WFS
capabilities document.
• The WFS returns an XML schema that describes the requested feature.
• Metadata about a specific entry.
• GetFeature: The client can ask the WFS to return a particular portion of any
feature data.
• GetFeature requests contain some property names of the feature and a Filter
element to describe the query.
• The WFS extracts the query and bounding box from the filter and queries the
feature databases.
• The results obtained from the DB query are converted the feature’s GML
format and returned to the client as a FeatureCollection.
11
Example WFS Capability Entries
ELEMENT NAME
DESCRIPTION
Name
A name the service provider assigns to the web feature service instance.
Title
Human-readable title to briefly identify this server in menus.
Abstract
Descriptive narrative for more information about the server.
Keyword
Contains short words to aid catalog searching.
OnlineResource
Defines the top-level HTTP URL of this service. Typically the URL of a "home page" for the
service.
Fees
Contains a text block indicating any fees imposed by the service provider for usage of the
service or for data retrieved from the WFS. The keyword NONE is reserved to mean no
fees.
AccessConstraints
Text block describing any access constraints imposed by the service provider on the WFS or
data retrieved from that service. The keyword NONE is reserved to indicate no access
constraints are imposed.
12
Sample Feature- CA Fault Lines
<gml:featureMember>
<fault>
<name>Northridge2</name>
<segment>Northridge2</segment>
<author>Wald D. J.</author>
<gml:lineStringProperty>
<gml:LineString
srsName="null">
<gml:coordinates>
-118.72,34.243 -118.591,34.176
</gml:coordinates>
</gml:LineString>
</gml:lineStringProperty>
</fault>
</gml:featureMember>




After receiving getFeature
request, WFS decodes this
request, creates a DB query
from it and queries the
database.
WFS then retrieves the
features from the database
and converts them into GML
documents.
Each feature instance is
wrapped as a
gml:featureMember element.
WFS returns a
wfs:FeatureCollection
document which includes all
featureMembers returned in
the query result.
13
`
WMS
le
ec
tio
n
Fe
a
ol
tur
eC
eC
oll
Ge
tF
ea
e
r
tu
r
tu
a
Fe
a
Fe
et
G
tur
e
Client
io
ct
n
•A WFS can serve multiple
feature types data.
•WFS returns the results of
GetFeature requests as GML
documents (Feature Collections).
•Clients may include other
services as well as humans.
s
ad
i l ro ]
a
b
R [a-
Railroads
WFS Server
Hi
River [a-d]
Bridge [1-5]
ry
SQL Query
ue
LQ
SQ
SQ
L
gw
ay
[1
2-
Q
ue
18
ry
]
Interstate
Highways
Rivers
Bridges
90
Conclusion and Other Topics

We have reviewed Community Grids Lab efforts to
build a GIS-compatible set of data grid services.
• This provides a unified architecture for grid execution and
data grid services.

Two other important topics:
• Managing messaging between components.
 NaradaBrokering project led by Dr. Shrideep Pallickara.
 www.naradabrokering.org
• Managing client environments
 Component-based portals: www.collab-ogce.org.
 Command line and scripting interfaces for web services:
www.hpsearch.org.
15
More Information

My email:
• [email protected]

Overview (submitted to ACES special issue of PAGEOPH).
• http://www.servogrid.org/slide/GEM/SERVO/ISERVO_ACES_PAGEOPH.doc

QuakeSim Project Page:
• http://quakesim.jpl.nasa.gov/

QuakeSim Portal:
• http://www.complexity.ucs.indiana.edu:8282.

QuakeTables Fault Database:
• http://infogroup.usc.edu:8080/public.html

Community Grids Lab GIS Grid Development:
•
•
•
•
http://www.crisisgrid.org/.
See particularly http://www.crisisgrid.org/html/servo.html for GML information.
See http://www.crisisgrid.org/html/wfs.html for more on Web Feature Service.
WMS and Information Services information coming soon.
16