Transcript No Slide Title

Application of c-squares spatial
indexing to an archive of
remotely sensed data
Tony Rees and Glenelg Smith
Divisional Data Centre + Remote Sensing Facility
CSIRO Marine Research, Australia
([email protected])
The starting point …
•
Archive of c.60,000 satellite
AVHRR scenes, 1986current, accumulating at
another 15+ per day
•
Region covered varies for
each scene
•
Metadata (satellite, orbit,
time parameters) and
computed polygon (e.g. 7
points per side) available for
each scene
•
Determining which scenes
include any given point, onthe-fly (to answer user’s
request) is a standard
requirement, but not trivial –
available methods may be
slow, or inaccurate, or both...
example in more detail ...
x
•
Example – may need to query which out of these four scenes includes
point “x” (in reality, stop/start points of scenes also vary, also other
scenes cross at different angle/s)
Options for spatial indexing
Bounding polygons
Bounding rectangles
•
•
Computationally cheap, rapid to query, no
GIS required
•
Typically a poor fit to actual data footprints
(typically these are not square, or even
nearly so) – many “false positives”
•
Can be problems with footprints which
cross the date line, or include a pole
•
Require a GIS system (or similar)
for spatial queries (polygon
overlays); can be slow to test
many thousands of scenes
Can be problems with footprints
which cross the date line, or
include a pole
Gridded (tile based) representation
•
Potentially simple to query, no GIS required
•
May be poor fit to actual data footprints if tile size is large (smaller tiles best,
however index gets larger – tradeoff required here)
•
Compromise approach – slightly inexact compared with polygon overlays, but
should be better performance for real-time queries (most computation has been
done in advance)
•
Problems with footprints which cross the date line, or include a pole, can be prehandled
C-squares basics
• Based on a tiled (gridded) representation of the earth’s
surface, at choice of 10 x 10, 5 x 5, 1 x 1, 0.5 x 0.5
degree tiles, etc. (0.5 x 0.5 degree tiles are used in this
example)
• Every tile has a unique, hierarchical alphanumeric ID
(incorporates the ID of all parent, grandparent, etc. in
every child tile ID)
• Dataset (=scene) extents are represented by a list of all
the tiles included in, or intersected by the footprint
• Spatial search comprises looking for one or more tile
IDs in the set associated with any dataset (= simple text
search).
(more details – see www.marine.csiro.au/csquares/ )
C-squares nomenclature – ten degree squares
(follows WMO
10 x 10º square
nomenclature)
Adaptive square size – 10 x 10, 5 x 5, 1 x 1, 0.5 x 0.5 degrees
(etc.) – produces quadtree-like efficiencies (“compaction”)
(this example
shows detail to 1degree square
level)
10 degree square:
3213:***
5 degree square:
3214:1**
1 degree square:
3315:100
- can reduce storage
requirement by up to
90%, cf. constantresolution encoding
A real-world example
NOAA-12 .. 21 Jun 2003 06:23
5 x 5 degree squares (99)
= 36 after “compaction”
1 x 1 degree squares
(1,982) = 515 after
comp.
10 x 10 degree
squares (28)
(base level of hierarchy,
cannot compact)
0.5 x 0.5 degree squares
(7,691) = 704 after comp.
0.5 x 0.5 degree squares
- detail
Metadata table
concept – 2 “search” tables only
Scene details (1 row per scene)
Scene <> c-square pairs
(600-1200 rows per scene)
Clickable
map
interface
Example search result
SQL for spatial search
(example for 0.5 degree search square)
select distinct A.scene_id, B.satellite, B.scene_date_time, B.image_location
from satdata_csq_all A, scene_info B
where (
(sqrsize = 0.5 and
(A.csquare = search_csq -- e.g. 3414:100:1 (0.5 degree square)
or A.csquare = substr(search_csq,1,8)||
':*'
or A.csquare = substr(search_csq,1,6)|| '**:*'
or A.csquare = substr(search_csq,1,4)||':***:*')
-- 1 level of compression
-- 2 levels of compression
-- 3 levels of compression
) -- (other supported search square size options inserted here)
and
and
and
and
order by
)
(startdate is null or B.scene_date_time >= startdate)
(enddate is null or B.scene_date_time <= enddate)
(sat = 'any' or B.satellite = sat)
A.scene_id = B.scene_id
B.scene_date_time, B.satellite;
Summary of steps involved
•
(1) Encoding – using custom polygon-to-c-squares
algorithm (first attempt: 3+ mins per scene; second
attempt: 30 seconds per scene to encode)
•
(2) Storage – first attempt: all scene/c-square pairs in
one large table, second attempt: partitioned by 10 x 10º
“bins” (100 smaller tables, for faster searching)
•
(3) Querying – using standard SQL and text matches
•
(4) (optional) Visualising – generating footprints, using
web call to multi-purpose c-squares mapper
More information: c-squares website http://www.marine.csiro.au/csquares/
CMR Satellite Data spatial search:
http://www.marine.csiro.au/remotesensing/csq-chooser.htm