Transcript Using Topic-Based Publish/Subscribe for Managing Real Time GPS Streams Marlon Pierce, Galip Aydin, Zhigang Qi Community Grids Lab Indiana University.

Using Topic-Based Publish/Subscribe for
Managing Real Time GPS Streams
Marlon Pierce, Galip Aydin, Zhigang Qi
Community Grids Lab
Indiana University
1
California Real Time Network
Continuous GPS Stations (CGPS) are depicted as
triangles while the Real-Time stations are
represented as circles. Image is obtained from
SOPAC GPS Explorer at
http://sopac.ucsd.edu/projects/realtime
Message Format
Network Data Rates
CRTN GPS
Site Positions
(9 Stations)
Entire SCIGN
Network (250
stations)
Time
RYO
ASCII
GML
1 second
1.5KB
4.03KB
48.7KB
1 hour
5.31MB
14.18MB
171.31MB
1 day
127.44MB
340.38MB
4.01GB
1 month
3.8GB
9.97GB
123.3GB
1 year
45.8GB
119.67GB
1.41TB
1year
1.23TB
16.18TB
160TB
How does one manage all the data generated by the
85 stations? How can you get just the data you want?
Note this is fundamentally different from traditional
request/response style Web Services.
RDAHMM: GPS Time Series Segmentation
Slide Courtesy of Robert Granat, JPL
GPS displacement (3D)
length two years.
Divided automatically
by HMM into 7 classes.
Features:
• Dip due to aquifer
drainage (days 120250)
• Hector Mine
earthquake (day 626)
• Noisy period at
end of time series


Complex data with subtle signals is difficult for humans to
analyze, leading to gaps in analysis
HMM segmentation provides an automatic way to focus
attention on the most interesting parts of the time
Use Case - GPS Sensors


GPS is used to identify long-term tectonic deformation and static
displacements. SCIGN has 250 Real-Time GPS Stations.
SOPAC GPS networks:





Our Architecture





8 networks for 80 stations produce 1Hz high resolution data.
Socket based real-time binary-RYO format access is available.
We developed filters to provide multiple format (RYO, ASCII, GML) realtime streaming access.
OHIO principle (a general principle required by DOD) and chain of filters.
Uses publish/subscribe based NaradaBrokering for managing real-time
GPS streams
Utilizes topics for hierarchical organization of the sensors
Deploys successive data filters ranging from format translators to data
analysis codes
Could potentially be used to run RDAHMM clones to monitor state
changes in the entire GPS network
We are partner in a pioneering project to use the real-time GPS data for
the first time in this context.
4
Processing Real-Time GPS Streams
ascii2gml
ryo2ascii
RYO
Ports
ascii2pos
7010
Raw Data
Scripps
RTD
Server
7011
NB
Server
ryo2nb
Single
Station
7012
Displacement
Filter
GPS Networks
RDAHMM
Filter
Raw Data
ryo2nb
ryo2ascii
ascii2pos
Station
Health
Filter
Single
Station
RDAHMM
Filter
/SOPAC/GPS/CRTN01/RYO
/SOPAC/GPS/CRTN01/ASCII
/SOPAC/GPS/CRTN01/POS
/SOPAC/GPS/CRTN01/DSME
A Complete Sensor Message Processing Path, including a data analysis application.
5
Application Integration with Real-Time Filters



RDAHMM
Station
Monitor
Filter records
Filter
records real-time
real-time
positions for 10
positionsand
minutes
for invokes
10 minutes
and calculates
RDAHMM
application
position
changes
which
determines state
changes
in theApplication
XYZ signal.
Graph Plotter
createsPlotter
Graph
visual Application
representation
creates
visual of the
positions.
representation
of the
RDAHMM output.
6
GIS Grid Usage Model – Earthquake Science



Supporting geophysical repositories and real-time
sensors is essential
To analyze a typical earthquake it is important to access
to precise measurements of the initial earthquakes and
aftershocks
To support earthquake forecasting and the time and
spatial positions of the forecasts



PI can be used with existing data
RDAHMM can be used with the real-time data
Earth Science field is moving from a previously data poor
field to a data rich world. We will have thousands of
sensors spread around the world. (i.e. GPS sensors, InSAR
satellites)
7
Recording and Replaying Sensor Streams






Filters can be used to record and replay scenarios,
such as Earthquakes in GPS case.
We developed RYO Recorder and RYO Publisher
Filters.
The RYO Recorder creates daily archives of the GPS
Streams.
RYO Publisher can be used to play daily or certain
segments of the records.
We replayed the 2004 Southern California
Earthquake using Parkfield GPS network archive
These filters are used in the performance and
scalability tests.
8
SensorGrid Performance Tests

Two Major Goals: System Stability and
Scalability
Ensuring stability of the distributed Filter Services
for continuous operation.
 Finding the maximum number of publishers
(sensors) and clients that can be supported with a
single broker.


Investigate if system scales for larger number
of sensors and clients.
9
Test Methodology
RYO
Publisher
1
NB
Server
1
2
3
RYO To
ASCII
Converter
4
Ttransfer = (T2 – T1) + (T4 – T3)
Simple
Filter




The test system consists of a NaradaBrokering server and
a three-filter chain for publishing, converting and
receiving RYO messages.
We take 4 timings to calculate mean end-to-end delivery
times of GPS measurements.
The tests were run at least for 24 hours.
GridFarm001-008 servers are used in these tests.
10
1- System Stability Test
System Stability Test
6
Time (ms)
5
4
3
2
1
0
0:00
1:00
2:00
3:00
4:00
5:00
6:00
7:00
8:00
9:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
The basic system with
three filters and one
broker.
 The figure shows
average results for
every 30 minutes.
 The average transfer
time shows the
continuous operation
does not degrade the
system performance.

Time Of The Day
Transfer Time
11
2 – Multiple Publishers Test
RYO
Publisher 2
Multiple Publishers Test
6
RYO
Publisher 1
2
1
Time Of The Day
Simple
Filter


Transfer Time
Standard Deviation
We add more GPS networks by running more publishers.
The results show that 1000 publishers can be supported with no
performance loss. This is an operating system limit.
12
22:30
21:00
19:30
18:00
16:30
15:00
13:30
12:00
10:30
9:00
7:30
0
6:00
RYO
Publisher n
4:30
Topic
1B
Topic
n
0:00
RYO To
ASCII
Converter
NB
Server
3
3:00
Topic
1A
4
1:30
Topic
2
Time (ms)
5
3 – Multiple Clients Test
1000 Clients
Multiple Clients Test
RYO
Publisher 1
40
35

Simple
Filter 2
900 1,000
Number
Time Of
OfThe
Clients
Day
Transfer Time
22:30
21:00
800
19:30
700
18:00
16:30
600
15:00
500
13:30
400
12:00
300
10:30
200
9:00
100
7:30
1
6:00
Adding Clients
4:30
20
20
15
15
10
10
5
5
0
0
3:00
Simple
Filter n
25
25
1:30
Topic
1B
Simple
Filter 1

NB
Server
0:00
RYO To
ASCII
Converter
30
30
(ms)
Time
Time(ms)
Topic
1A
Standard Deviation
We add more clients by running multiple Simple Filters which subscribe
to the same ASCII topic.
The system can support as many as 1000 clients with very low
performance decrease.
13
Extending Scalability




The limit of the basic system appears to be 1000
clients or publishers.
This is due to an Operating System restriction of
open file descriptors (1024 for Red Hat Linux) which
can be increased by changing OS parameters.
To overcome this limit we create NaradaBrokering
networks with linking multiple brokers. NB supports
scalable linkage of the brokers for building tree like
architectures.
We run 2 brokers to support 1500 clients.

Number of brokers can be increased indefinitely, so we can
potentially support any number of publishers and
subscribers.
14
4 – Multiple Brokers Test

RYO
Publisher
RYO To
ASCII
Converter


Simple
Filter 1
Topic
1A
NB
Server
1
Simple
Filter 2
Topic
1B
Simple
Filter
750
Simple
Filter
751
NB
Server
2
NB
Server
2
Simple
Filter
752
Topic
1B



NaradaBrokering allows creation of
Broker networks.
We create a two-broker network.
Messages published to first broker can
be received from the second broker.
We take timings on each broker.
We connect 750 clients to each broker
and run for 24 hours. We chose 750
clients to stay well below the saturation
limit.
The results show that the performance
is very good and similar to single broker
test.
Simple
Filter
1500
15
4 – Multiple Brokers Test
Multiple Clients Test
Broker 2
40
30
35
25
30
20
15
25
20
15
Time Of The Day
Transfer Time
Standard Deviation
22:30
21:00
19:30
18:00
16:30
15:00
13:30
12:00
10:30
9:00
0
7:30
0
6:00
5
4:30
5
3:00
10
1:30
10
0:00
1:00
2:00
3:00
4:00
5:00
6:00
7:00
8:00
9:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
17:00
18:00
19:00
20:00
21:00
22:00
23:00
Time (ms)
35
0:00
Time (ms)
Multiple Clients Test
Broker 1
Time Of The Day
Transfer Time
Standard Deviation
16
Real-Time Filters Test Results
The RYO Publisher filter runs at 1Hz and
publishes 24-hour archive of the CRTN_01 GPS
network, which contains 9 GPS stations.
 The single broker configuration can support
1000 clients or publishers (GPS networks 9000 individual stations).
 The system can be scaled up by creating
NaradaBrokering broker networks.
 Message order was preserved in all tests.

17
Acknowledgement




Galip Aydin: Designer and implementer
ZhiGang Qi: SensorGrid Performance Tests
We thank Prof. Yehuda Bock and his group at SIO for
their help with real-time GPS data streams.
Robert Granat, JPL: RDAHMM developer
18