Transcript Concurrent Web Map Cache Server Zao Liu, Marlon Pierce, Geoffrey Fox Community Grids Laboratory Indiana University.

Concurrent Web Map Cache
Server
Zao Liu, Marlon Pierce, Geoffrey Fox
Community Grids Laboratory
Indiana University
Introduction
•
•
•
Geographical Information Systems combine
online dynamic maps and databases.
Many GIS software packages exist
GIS servers around state of Indiana
–
–
–
•
ESRI ArcIMS and ArcMap Server (Marion,
Vanderburgh, Hancock, Kosciusco, Huntington,
Tippecanoe)
Autodesk MapGuide (Hamilton, Hendricks, Monroe,
Wayne)
WTH Mapserver™ Web Mapping Application (Fulton,
Cass, Daviess, City of Huntingburg) based on several
Open Source projects .
These are not compatible
Map Server Federation
• Integrating GIS map servers is not trivial
• Our solution: create a virtual map server to act as an agent server
– Translates map requests from generic format to the format expected by
the specific map server.
– Provides a common language and programming interface for
constructing clients
• The agent server by itself will work but performance is not good
– Must wait for slowest server to respond
– Failure prone: a county server may not respond at all
– Adds additional overhead for combining images
• Combining the agent server with a caching server solves these
problems.
– Caches images for greater performance
Agent Server Architecture
County Server
Agent server
Caching Server
• The agent server runs offline to harvest map images from county
map servers.
– Images are stored as tiles.
– Tiles at county boundaries may be combined for greater storage and
performance efficiency.
• Clients connect to the cache server instead of the agent server.
• The cache server constructs the requested image from pre-fetched
tiles.
– Inspired by Google Maps approach
– Will enable more interactive clients (so-called AJAX programming)
• Image construction may be parallelized/multi-threaded for greater
performance.
– Potentially takes advantage of new multi-core server architectures from
Sun, Intel, and AMD.
Tiling Example
Agent server requests entire county
maps for a particular zoom level and
then breaks up into tiles.
Tiling and Caching at County
Boundaries
Bounding box requests across boundaries have
many empty tiles.
Marion County
Hancock County
Removing these empty tiles decreases storage
requirements and increases cache server performance
The combined map
Caching and Tiling Layers
• Map servers typically
contain base maps and
optional layers
– Parcel boundaries, roads,
and township boundaries
are layers.
• We cache each layer
separately.
• Layers and base maps
are combined
dynamically using Java
Advanced Image
libraries.
– Common techniques
Google Maps – WMS Integration
•
Currently Google Map supports attractive
street map and aerial photo.
• County map server supports more geographic
information in details.
• Issues
-- Map Re-projection (Google Map and Sattellite
are based on Mercator projection and WMS
are based on EPSG:4326)
-- Caching mechanism (should be the same as
Google tiling method)
Map Re-projection
Combine maps (no re-projection) VS Combine maps (re-projection)
Combine Google map with county parcel data
Tradeoffs of Caching
• Cached images must be stored
somewhere.
• Currently, three counties (Hancock, Marion,
and Cass) are cached at 11 different zoom
levels.
– Photo images, layers
– Takes NN GB of storage
Caching the Entire State
• Takes about 500-600 GB to store the entire state to
zoom level 10 this way.
– There are 6047804 tiles for zoom levels 0-10, 15131920 tiles for
0-11 levels
– There are ~10 layers for each scale
• Aerial photo layer tiles take 60 KB
• Other layers (parcels, roads) are much smaller: 36 KB for all
remaining 9 layers per tile
• So we need 96KB * 6047804 tiles to store all map data
– This is large but possible.
– Current commercial servers hosts like Sun T2000 can have 1 TB
external (RAID) storage.
• We can easily spread our caching server over multiple
hosts to store even higher magnification scales.
• Efficient tiling storage can save disk space.
Summary of Contributions
• Development of agent server to pre-fetch map
images from county map servers.
– Stores images as tiles.
– Removes redundant/empty tiles.
– Supports ESRI and OGC servers
• Development of caching server
– Provides a uniform mechanism for clients to interact
with different map servers.
– Increases performance and reliability
• Don’t have to go to source map servers for every request.
– Will enable more interactive clients
• Google Map-like interaction
Next Steps
• Add more counties
– If county uses ESRI or OGC map server,
current agent plugins can be used.
• Formalize programming interface using
Web Service standards
– WSDL and SOAP
• Investigate scalability and performance
issues
Additional Slides
Results & demo
• Performance improvement
Architecture
CountyGisServer
• XMLFetcher
• MapFetcher
MapFetcher
XMLFetcher
Buffer
DB
Architecture – XMLFetcher
• XMLFetcher
– Request info from servers
• fetches XML files not real data
– Save xml into buffer for MapFetcher use
• each represents a 1024 * 1024 image
– Send request to several servers for a
boundary map
• With same coordinates
Architecture – XMLFetcher
Architecture – MapFetcher
• MapFetcher
– Query database
– fetch real data from servers
– Store maps and relate info on local disk
– Tell DBManager to update the database
Tile Reconstruction
• Projection Uniform
• Layer overlap
Tile Reconstruction-- Example
Parcel layer (combined with tiles)
Tile Reconstruction-- Example
• Aerial photo layer