Transcript Slide 1

Reachability Analysis for Discrete
Time Stochastic Hybrid Systems
Saurabh Amin
Alessandro Abate
Shankar Sastry
UCBerkeley
http://chess.eecs.berkeley.edu/
Introduction
Computational Results
Stochastic hybrid systems (SHS) can model uncertain
dynamics and stochastic interactions that arise in
many systems. An important problem in SHS theory is
that of probabilistic reachability
Probabilistic Reachability Problem
• What is the probability with which the system can
reach a set during some time horizon?
• (If possible), select a control input to ensure that the
system remains outside the set with sufficiently high
probability
• When the set is unsafe, the problem becomes a
quantitative safety verification problem. In this case,
find the maximal safe sets corresponding to different
safety levels
Discrete Time Stochastic Hybrid
System (DTSHS)
DTSHS definition:
Where is the set of modes, the map defines the
dimension of the continuous state space of these
modes, and are the transition and reset control
spaces, and , , and are continuous, discrete, and
reset stochastic kernels respectively
Dynamics of thermostat system
Continuous dynamics
Off
On
Continuous and reset transition kernels
Off
On
Optimal control policy for three
different safe sets
Executions generated by optimal
policy for the three safe sets
Discrete transition kernel
“Switch” action
“Don’t switch” action
Stochastic Reachability
,
Consider Markov polices
Assume complete observability and finite time horizon
Reach probability is the probability that the execution of
associated with policy
and initial distribution
will enter set during time
Probabilistic safe set is the set that guarantees safety probability
:
for policy
Maximal probabilistic safe Sets for different safety levels
Backward Reachability Computations
Since
we have
DTSHS as controlled Markov process
For computing reach probability for fixed Markov policy
define the functions
by
State space
Control space
,
Controlled transition kernel
which is
if
and
if
Motivational example
Thermostat
ON
OFF
Then,
For initial mode “OFF”
and so,
Maximal Probabilistic Safe Set Computation
Conclusions and Future Work
For safety level
For controlled DTSHS
the maximal safe set
Dynamic programming recursion
Define the functions
Trivial switching control law
by
and so,
Then,
• Proposed a model for controlled discrete time SHS suitable for optimal control
and reachability analysis
• Interpreted the safety verification problem in terms of stochastic reachability
notion
• Developed a dynamic programming based approach for computing probabilistic
maximal safe sets and the optimal feedback policy
• Applied the proposed methodology on a simple example and presented
computational results
Future work
Existence of optimal policy is shown and
February, 23 2005
For initial mode “ON”
• To address the problem of reachability analysis for continuous time SHS
• Apply the research to probabilistic safety verification problem in more challenging
applications such as air traffic control systems
2006 Berkeley EECS Annual Research Symposium (BEARS)