bas2003 3870
Download
Report
Transcript bas2003 3870
Modeling Adaptive Robot Swarms
Not an Inverse Problem
Kristina Lerman
Aram Galstyan
USC Information Sciences Institute
Why Analyze Adaptive Systems?
To predict
Behavior of the system under new conditions
Aspects of the system not studied experimentally
Large numbers of robots
Dynamic environments
To control
Find parameters that optimize system performance
Optimal number of robots for the task
Find parameters that prevent instabilities, etc.
To understand
How individual robot characteristics affect collective behavior
How system size affects performance of the system
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Modeling and Analysis Tools
Microscopic (discrete) models
describe/model individual robot’s behavior
Equations of motion approach
Explicitly account for all interactions
Solve to obtain trajectories
E.g., pheromone-based trail formation
(Schweitzer et al., 1997)
Microscopic simulations
Abstract individual robot properties
Probabilistic models (Martinoli et al., 1999)
Robot’s actions modeled as series of stochastic events
E.g., collaboration in robots (Ijspert et al., 2001)
Others: cellular automata, molecular dynamics, etc.
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Modeling and Analysis Tools (cont)
Macroscopic (continuous) models
describe collective behavior of groups of robots
Finite difference equations
Collaboration in robots
Synchronous
(Martinoli & Easton, 2003)
Rate Equations
Continuous limit of finite difference equations
Alternatively, derived from Stochastic Master Equation
Collaboration (Lerman et al., 2001) and foraging (Lerman &
Galstyan, 2002) in robots
Asynchronous
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Rate Equations
We will use macroscopic models – Rate Equations
– to study robots
Rate Equations describe average collective
behavior, not a specific experiment
More computationally efficient than microscopic models
Directly describe experimental observations
describe results averaged over many experiments
They are usually phenomenological
Not derived from first principles
Are easier to construct because you don’t have to explicitly
model many individual details
Microscopic details appear only in parameters of models
Can be derived from stochastic microscopic description
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
What We Can Model Now
Types of robots modeled
Reactive robots
Perception and action are tightly coupled
No memory or use of historical information
Can use timers to trigger actions
Adaptive robots
Change their behavior in response to environmental changes or
actions of other robots
These are simple robots
No intentionality
No abstract representations
Distributed system
collective behavior arises out of local interactions among robots
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Why Amenable to Analysis
Individual robots can be modeled as stochastic
processes
Individual robot’s behavior subject to
External forces
Noise
fluctuations and random events
Other robots with complex trajectories
may not be anticipated
Can’t predict which robots will interact
Errors in sensors and actuators
Randomness programmed into controllers
e.g., avoidance
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Our Results
A framework for modeling collective behavior in
reactive and adaptive robot systems
Derived equations from theory of stochastic processes
“Recipe” for creating application-specific models from individual
robot controllers
Models of distributed robot systems
Collaborative stick-pulling
Qualitative agreement with experimental/simulations results
Analytic form for critical parameters
Foraging
Quantitative agreement with simulations results
Optimal group size and its dependence on individual robot
parameter
Dynamic task allocation
Individual robot transition rules for stable steady state distribution
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Adaptation in Robot Swarms
Adaptation is crucial for robotic swarms
functioning in unstructured dynamic
environments
Adaptation allows robots
to change their behavior in response to
environmental changes or actions of other
robots
to improve the overall performance of the
system
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Memory as a Mechanism for Adaptation
Memory-based adaptation
Robot makes observations; stores them in memory
Uses observations to estimate the state of the
environment and other robots
Modifies its behavior accordingly
Adaptive robots using memory of length m
to make decisions about future actions
can be represented as a generalized
Markov process of order m
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Adaptive Robots as Markov Processes
Individual robot probability distribution
p(n,t) = probability robot is in state n at time t
Generalized Markov property: robot’s state at
time t+Dt depends on its state at time t, and
history of past m-1 states
h={(n1,t), (n2,t-Dt),…, (nm,t-(m-1)Dt)}
p(n, t Dt | h) p(n, t Dt | n, t ; h) p(n, t | h)
n
Change in probability density
Dp(n, t ) p(n, t Dt ) p(n, t )
[ p(n, t Dt | h) p(n, t | h)] p(h)
USC Information Sciences Institute
October 4, 2003
h
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Master Equation for Adaptive Systems
In the continuum limit, Dt 0
dp (n, t )
W (n | n, t ; h) p(n, t | h) p(h)
dt
h n
W (n | n, t ; h) p(n, t | h) p(h)
h n
with transition rates
p (n, t Dt | n, t ; h)
W (n | n, t ; h) lim
Dt
Dt 0
Similar to the Stochastic Master Equation that
describes evolution of physical systems
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
From One to Many
Collective robot state
If robots are independent and indistinguishable
Collective configuration described by n={N1,…, NL}
Nj = number of robots in state j
P(n,t) = probability system is in configuration n at t
Master Equation for P(n,t) specifies how the entire
system evolves in time
dP(n, t )
W (n | n, t ; h) P (n, t | h) P (h)
h n
dt
W (n | n, t ; h) P (n, t | h) P (h)
h n
However, ME is often difficult to formulate and solve – it is
difficult to define correct probability distribution for a system
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Rate Equation
Instead, we work with the Rate Equation
Derived from the Master Equation
“First moment” of the ME
Describes how the average number of robots in
state k changes in time
d N n (t )
W (n | n; t )
n
dt
h
N n W (n | n; t )
n
h
Nn
where <…>h denotes average over histories
No need to know exact probability distributions
Used in Ecology, Population Dynamics
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Dynamic Task Allocation
Jones & Mataric
USC Information Sciences Institute
October 4, 2003
Task
Goal
Mechanism
Robots are allocated to collect
red or green pucks
Dynamically achieve an
appropriate division of
labor
• Robot makes local
observations and adds
them to memory
• Each robot estimates the
proportion of pucks and
robots in the environment
(from memory), and
switches state accordingly
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Modeling Adaptive Task Allocation
dN R (t )
a R (t , h) N G (t ) a G (t , h) N R (t )
dt
dM R (t )
b R M R (t ) N R m R 0
dt
NR|G(t) number of robots in Red|Green state
MR|G(t) number of Red|Green tasks
aR|G(t,h) rate robots switch to Red|Green state
bR|G rate robots complete Red|Green tasks
mR|G rate at which new Red|Green tasks are added
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Transition Rates
Transition rates depend on
nR,obs=NR,obs/(NR,obs+NG,obs) : observed density of Red robots
mR,obs= MR,obs/(MR,obs+MG,obs) : observed density of Red pucks
Mathematical form of transition rates
Steady-state (SS):
dnR/dt=0=aR nG-aGnR
Desired SS solution: nR,SS= mR
Intuitive guess: aR=f(nR-mR)
Will lead to desired SS solution only when f(nR-mR)=0
Informed by analysis:
aR=(1- mR)f(nR-mR)
guarantees desired steady-state is satisfied non-trivially
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Mathematical Form for f
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Intuitive vs Informed Transition Rates
Simulations results with different transition rates
Intuitive (power f ):
aR=f(nR-mR)
Informed (power f ):
aR=(1- mR)f(nR-mR)
time
USC Information Sciences Institute
October 4, 2003
Courtesy of C. Jones
ISI
Modeling Adaptive Robot Swarms
K. Lerman
History and Transition Rates
…
h
h
Robot’s nRn,obs
h
Rn,obs
h
memory mR,obs
hR ,obs
mR (t 3D )
nR (t D )
mR (t D )
…
…
…
nR (t 3D )
t
n1R,1obs nR0 ,0obs
n obs nR,0obs
1 Rn,1R
R ,obs
mR,1obs,obsmR0 ,n0obs
mR,1obs mR,0obs
mR,obs mR,obs
mR ,obs
h
mR ,obs
All
robots
t-D
…
t-|h|D
nR (t )
mR (t )
a R (t , h) (1 mR (t , h)) f (nR (h) mR (t , h))
1 h1
nR (t , h) nR (t iD )
h i 0
1 h1
mR (t , h) mR (t iD )
h i 0
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Time Delay Equations
dnR (t )
a R (t , h)nG (t ) a G (t , h)nR (t )
dt
1 h1
a R (t , h) (1 mR ) f nR (t iD) mR
h i 0
1 h1
a G (t , h) mR f mR nR (t iD)
h i 0
Initial conditions
nR (t 0) 1, nG (t 0) 0
mR (t ) mR
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Dynamics of Red Robots
Linear f
USC Information Sciences Institute
October 4, 2003
Solutions show
oscillations
characteristic of
delay equations
Solutions
eventually relax to
puck distribution
Magnitude of
oscillations and
relaxation time
depend on size of
history window
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Solutions for Different f Same h
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Jones & Mataric
Conclusions
Created a model of collective dynamics based on
theory of stochastic processes
Applied formalism to distributed robotic systems
Reactive robots
Adaptive robots
Collaborative stick-pulling
Foraging
Dynamic task allocation
Results
Theoretical predictions agree at least qualitatively with results of
experiments and simulations
Analytic results not obtainable by other methods
Insights into robot design (form for transition rates)
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Future Work
Beyond the Rate Equation
Take into account fluctuations
Noisy observations
Formulate and solve the collective Master Equation
Appropriate form for probability distribution function for dynamic
task allocation application
More realistic models
Don’t coarse-grain behaviors
Automatic model construction
Other systems – new challenges
Self-reconfigurable robots
Nano-robots
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Roadmap to Theory
Starting with an individual robot
Derive stochastic Master Equation
ME describes how robot’s state changes in time
State=action or behavior the robot is executing
Make transition to a multi-robot system
Derive collective Master Equation
describes how configuration of the system evolves in time
ME is often difficult to formulate and solve
Instead, work with the Rate Equation
“Mean” or “First Moment” of the ME
Practical “recipe” for constructing the Rate Equation
from individual robot controller
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Representation of Reactive Robots
Finite state automata used to represent
individual reactive robots (Arbib et al., 1981)
State = behavior; transitions between states
Example: simplified foraging diagram
Collective behavior is captured by the same FSA
Each robot in exactly one of finite number of states
State = number of robots executing that behavior
start
searching
Reach
home
homing
USC Information Sciences Institute
October 4, 2003
Gripper
closed
Puck
detected
pickup
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Coarse-graining
search
Avoid
obstacle
search
Detect
object
search
Avoid
obstacle
• Coarse-graining reduces the complexity of the model
• Helps construct a minimal model that explains experiments
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
A “Recipe” for Rate Equations
start
searching
homing
pickup
dN s
f s ( E , N s ) N s f h ( h , N ) N h
dt
dN p
f s ( E , N s ) N s f p ( p ) N p
dt
Nh N Ns N p
Initial conditions: Ns(t=0)=N, Nh(0)=0, Np(0)=0
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman
Transition Rates
Transition is triggered
By a stimulus
Obstacle, another robot in a particular state, location (e.g., home)
By a timer
Turn in a random direction for x seconds
Calculating transition rates
Calculated under assumptions
Triggers are uniformly distributed in space
Robots encounter triggers randomly
Estimated from data by
Calibration
Run experiment or simulation for a single robot in an arbitrarily
complex environment and measure relevant parameters
Fitting
Fit the model to the data
USC Information Sciences Institute
October 4, 2003
ISI
Modeling Adaptive Robot Swarms
K. Lerman