Transcript MonetDB - RGI OTB

A ‘movingpoint’ type for a DBMS
Wilko Quak - TUDelft
2008-11-06 Dagstuhl
1
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
Overview
• Clarification
• Design considerations for movingpoint type
• Implementation in MonetDB
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
2
Moore’s Law makes some problem
go away
50% p/year:
#points in
laser scan
- cpu speed
#available
- mem size
GPS
logs
- mem
bandwidth
- disk
bandwidth
1% p/year:
- mem latency
10% p/year:
#cadastral
parcels
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
- disk latency
Moving Point Type
3
Examples of
moving points
data
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
4
What do I want with the data?
• Can I find all occurences of missing data because
someone used subway?
• What is the optimal distance between busstops to get
people to the trainstation as fast as possible?
• Is that man smiling?
• Flocks and other patterns?
• More……?
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
5
Requirements
• Should work with all kinds of data
• Should be extensible (to moving region, or dynamic
integer??? (orthogonal?))
• Queries should be understandable
• Should work seamlessly together with
point/line/polygon + datetime
• I want to store my original measurements in a
reproducably and compact way
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
6
Two possible mappings of movinpoint
type to DBMS:
create table person(
name string,
location point dynamic continuous,
salary integer dynamic discrete);
This is ortogonal but
requires a highly
extensible DBMS to
implement
This is not that bad
and is easier to do
in existing DBMS
create table person(
name string
location movingpoint);
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
7
Things that should be efficient
•
•
•
•
Range queries in time
Range queries on location
Nearest neigbour search (Fréchet distance)
Joins
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
8
Debatable Issues
• What to do with accuracy. (Current implementations of
point line an polygon don’t have it. Do we miss it?)
• More interpolation types than: linear or constant.
• Do we want a multi-scale type?
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
9
MonetDB
A novel spatial column-store DBMS
Martin Kersten - CWI
Niels Nes - CWI
Wilko Quak - TUDelft
Maarten Vermeij - TUDelft
2008-11-06 Dagstuhl
10
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
MonetDB design considerations
•
•
•
•
•
•
•
•
Multi-model database kernel support
Extensible data types, operators, accelerators
Database hot-set is memory resident
Simple data structures are better
Index management should be automatic
Do not replicate the operating system
Optimize when you know the situation
Cooperative transaction management
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
11
MonetDB - Physical data organization
• Binary Association Tables
ID
10
11
12
13
OID
ID
100
101
102
103
104
OID
10
11
12
13
14
Day
Discount
4/4/98
0.195
9/4/98
0.065
1/2/98
0.175
7/2/98
0
Day
100
4/4/98
101
9/4/98
102
1/2/98
103
7/2/98
104
1/2/99
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
OID
Moving Point Type
Discount
100
0.195
101
0.065
102
0.175
103
0
104
0.065
12
MonetDB product family
End-user application
SQL
JDBC
ODBC
XQuery
Python
Perl
PHP
C-mapi lib
MAPI protocol
Monet
kernels
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
13
MonetDB
heap
layout
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
14
Redundancy
ID
X
Y
Z
T
1
1215324
4341572
3.14159
2006-06-02:16.47
2
1215327
4341579
3.2
2006-06-02:16.57
3
1215400
4341590
10.0
2006-06-02:17.07
4
1215321
4341706
11.0
2006-06-02:16.17
This can be compressed away in heap
structure
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
15
Implementation options for heap
structure
1. Implement as BLOB with x1,y1,z1,t1,x2,y2,z2,t2,…
- Similar to polyline implementations.
- Index with rtree index on x,y,z,t
- Is reasonably efficient for small objects.
2. Build a kinetic structure in the heap
- Will be efficient for querying
- Compression schema might be tricky
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
16
Conclusions / Discussion
• DBMS support for multipoints will make querying
collections of moving points easier and more efficient.
• MonetDB looks like a good option for implementation.
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
17
MonetDB extension mechanism for
types
2008-11-06
OTB Research Institute for Housing, Urban and Mobility Studies
Moving Point Type
18