Transcript Cooperating Sentient Vehicles for Next Generation Automobiles

Project IST-2000-26031
Middleware Platform for Sentient
Computing Applications
Thirunavukkarasu Sivaharan,
Maomao Wu, Gordon Blair,
Adrian Friday, Paul Okanda.
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Computing Department,
Lancaster University, UK
2nd MiNEMA Closed Workshop@
Lancaster, 1st Dec 2004
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Overview of Presentation
 Introduction
 Sentient Objects
 Research Challenges &
Component Frameworks
 Middleware Architecture
 Sentient Vehicle Demonstrator
 Conclusions
2
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Introduction(2)
EU FET Project : CORTEX
–
–
–
–
Universidade de Lisboa (Portugal)
Lancaster University (United Kingdom)
Trinity College (Ireland)
Universität Ulm (Germany)
Aims
– Middleware support for constructing distributed mobile
proactive applications based on real-time sentient objects
– Proposes sentient object model to support the construction
of mobile, context aware, decentralised ,autonomus
,proactive and collaborative applications such as intelligent
vehicles and smart buildings.
– A middleware for networked embedded systems
3
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Sentient Object Model(1)
4
Lancaster University
C
Sentient
Object
P
Event
Production
– System consists of
environment and a set of
sentient objects
– Sentient objects are capable of
independently sensing the
environment, derive context
and infer autonomous actions
– Sentinet objects communicate
using event channels to
establish higher level context
and thus cooperate with each
other
Event
Consumption
 Sentient Object Model
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Sentient Object(2)
Sensor
Actuator
Consume
Sensory
Capture
and
Fusion
Context
Inference
Representation
Engine
Sensor
Actuator
Sentient Object
Event
5
Produce
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Autonomous sentient vehicle application
in MANET
 Autonomous navigation of vehicles from a
source to destinations
 Cooperating vehicles in MANET
 Context aware vehicles
6
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Some of the research challenges
addressed
 Suitable Communication Model for MANET
 Routing in mobile ad-hoc environment
 Context-awareness
 End-to-End QoS and Fail safety
 Run time and deployment time reconfigurations
7
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Component Framework based Reflective
Middleware
 Publish-Subscribe Component framework (CF)
 Multicast CF
 Context CF
 Resource Management CF
8
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Why Component Framework based
Middleware Platform?
 Middleware is engineered as family of Component
frameworks (CF) using Reflection and component
technology
 Each CF addresses specific research areas
 Component Frameworks are highly configurable and
dynamically reconfigurable (with the granularity of a
component)
 Clear separation of concerns
 Adaptable to diversity of CORTEX applications
 Reduction of memory footprint
 CFs are implemented using Lancaster’s OpenCOM
reflective component technology
9
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Middleware Architecture
Sentient
Objects
M
I
D
D
L
E
W
A
R
E
Sentient
Objects
Context CFSensor Fusion
Inference Engine
Programming Interfaces
Publish-Subscribe CF- (for MANET)
Group Communication CF-( Ad-hoc Multicast )
Payload Channel
Timely Computing
Base
TCB control channel
WLAN 802.11b (ad-hoc),
Windows CE
Middleware Configuration for MANET
10
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Publish-Subscribe CF(1)




Communication model inspired by STEAM
Implicit event model
Sender & receiver based event filtering
Subscription Language supports subject, content &
context based event filtering
 Supports distance based context filtering & extensible to
other contexts
 XML based generic events
 Events transported via selectable Multicast protocol
11
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Publish-Subscribe CF(2)
ISubscribe
IPublish
IDispatch
Subscrib
er
Publisher
ISOAPMessaging
IFilter
Filter
SOAP
Messagi
ng
Dispatc
her
IFilter
Notifier
IApplicationNotify
Filter
ISOAPTransport
SOAPtoMult
icast
Receptacle
Interface
IMulticast
Multicast
12
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Multicast CF
 Underlying event Routing Protocol is based on
multicast
 The multicast protocol for ad-hoc networks is a
probabilistic, stateless and multi-hop protocol
 We offer this service in the form of a component
framework.
Shared
memory
based IP
Multicast
Probabilistic
Multicast
13
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Context CF (1)
 Sensor capture and fusion
– Multivariate Gaussian modelling
– Bayesian networks
– Dead-reckoning
 Inference engine
– A program that reasons about a set of rules (a knowledge base)
in order to derive an output.
– The knowledge is encoded as a set of production rules, contexts
are represented as “fact”.
– CLIPS – C Language Integrated Production System, its internal
implementation is based on RETE net.
14
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Context CF (2)
 CLIPS rule sample
(defrule rule-obstacle-near "CLIPS rule for obstacle near"
(car-id (id ?id))
?f1 <- (obstacle (distance near))
=>
(retract ?f1)
(publish ?id stop)
)
 The paradigm facilitates uniform treatment of both
context and QoS
– Rules to trigger adaptations and actuations based on changes in
measure of QoS data
 CLIPS DLL and OpenCOM component for WinXP and
WinCE
15
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
End-to-End QoS Management and Fail
Safety- Timeliness requirement
 How can this be achieved?
– Enforcing timely perceptions of the environment and
timely actuations on it.
– Which means timely event delivery and awareness of
QoS of the event channels used for inter-sentient
object communication
 The key issue in uncertain and highly dynamic
environments is that timing bounds for
distributed actions may be violated because of
timing failure
16
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
End-to-End QoS Management and Fail
Safety-Timeliness Requirements
 We model the uncertainty of timely event dissemination via event
channels using a dependable timing failure detection service.
 This service is provided by University of Lisboa’s Timely Computing
Base (TCB)
 TCB facilitates to construct distributed event channels with timing
bound specification
 This enables publisher or subscriber to be aware of the timing
failures of event channels
 Thus providing awareness of timing failure probability for a given
required coverage
 Fail safety is achieved by switching to fail-safe state as soon as QoS
specifications are violated.
17
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Autonomous Sentient Vehicles
Demonstrator
 Two Sub problems
– Cooperative behaviour without human control
– Autonomous vehicle navigation from a given source to
pre-determined destination
 Vehicles Objectives
– Travel along a given path( virtual circuit-VC) defined
by set of GPS waypoints and bearings.
– Every vehicle that travels on the VC cooperate with
other vehicles to avoid collisions and travel safely
– Obey external roadside traffic lights.
– Give way to pedestrians who cross the road.
18
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Location aware Cooperating Sentient
Satellites
Vehicles
IEEE 802.11b(ad-hoc)
---Event Channel--CarControlChannel
Car A
OC BEHIND
Car B
4m
OC CLOSE( 4m)
OC FAR(4- 10m)
OC BEHIND
Other car’s location context
w.r.t car A
19
OC VERY FAR
OC CLOSE
Other Car’s location context
w.r.t Car B
OC – Other car
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Pedestrian detection
 Obstacle Sensing Service: Consumes raw
ultrasonic sensor data and fuses using a suitable
algorithm (reliable, timely-unreliable, Gaussian,
…) to derive higher level obstacle distance
context such as NEAR , FAR , NOOBJECT.
Ultrasonic
sensors
20
Ultra sound
waves
Lancaster University
Non event
publishing
obstacle
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Example: The Car Sentient Object & Context CF
Speed
Actuator
Ultrasonic sensor
Obstacle
Sensing
Service
Ultrasonic
Fusion 1
Ultrasonic
Fusion 2
Steer
Actuator
Inference
Service
GPS sensor
Consume
Sentient object
Location
Sensing
Service
GPS
Fusion 1
GPS
Fusion 2
Produce
CLIPS
Component
Inference
Engine
Sentient object
Digital Compass
sensor
Direction
Sensing
Service
Compass
Fusion 1
Interface
Component
receptacle
21
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Sentient Vehicle Test Bed
22
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Cont’d
23
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Cont’d
24
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Demo Settings
25
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Waypoint 3
Waypoint 2
Traffic Light
Waypoint 4
Waypoint 1
Virtual Circuit
26
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Demo Video
27
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Concluding Remarks
 The sentient object model
– has proved to be valuable programming abstraction for the development
of real-time, cooperative, context-aware applications.
 The component-Framework based Middleware approach
– offers benefits of flexible configuration and reconfiguration of the
middleware components
 The middleware architecture
– also provides the management of non-functional concerns such as
timeliness and reliability properties.
 Our middleware is reusable
– we are keen to investigate the generality of our approach by applying
our middleware to other application domains involving embedded
autonomous components.
28
Lancaster University
CO-operating Real-time senTient objects:
architecture and EXperimental evaluation
Thank You
Questions
29
Lancaster University