Transcript No Slide Title

Recognizing Teamwork Activity in
Observations of Embodied Agents
Linus J. Luotsinen
School of Electrical Engineering and Computer Science
University of Central Florida
B. Sc. University of Dalarna, Sweden, 2002
M. Sc. University of Central Florida, 2004
Ph.D. Dissertation Defense
November 06, 2007
Committee Members
 School of Electrical Engineering and Computer Science
– Dr. Ladislau Bölöni
– Dr. Avelino Gonzalez
– Dr. Kenneth Stanley
 Department of Statistics and Actuarial Science
– Dr. Liqiang Ni
Outline
 Introduction
Covered in candidacy
 Data Acquisition and Knowledge Engineering
 Teamwork Activity Recognition using Hidden Markov Models
 Team-Oriented Feature Extraction
 Role-Based Teamwork Activity Recognition
 Tuning the Performance of Teamwork Activity Recognition
 Conclusions
Introduction
 Recognize teamwork activity in a stream of positional agent
traces, and annotate them with the recognized actions
 Video demonstration
Team merge (t=0)
Convoy (t=100)
Goal (t=200)
Applications
 Surveillance
– Recognize multi-agent activity in surveillance video feeds
 Training
– Identify discrepancies and deviations from the actions
performed by an expert team
 Smarter agents
– Model the opponent team to imitate or countermeasure its
actions
 Automated annotation
– Automatically index large databases for fast content retrieval
 After Action Review
 Digital Video Recorder
Challenges
 Observation noise
– Position traces can be distorted by inaccuracies in sensors and
localization algorithms
 Alignment problems
– Movement performed at different location and orientation
 Scaling problems
– Movement performed at different physical scale
 Temporal scaling
– Movement happens slower or faster in time
 Terrain distortion
– Movement distorted because of adaptation to the terrain
Challenges
 Movement variants
– Movement in alternative ways that map to the same label
 Uncertainty regarding the role of the agents in the team
 Role count variants
– Movement with different number of agents in the team
 Agents changing their roles during the team action
Outline
 Introduction
 Data Acquisition and Knowledge Engineering
– Software tools we developed
– Datasets we acquired, segmented and labeled
 Teamwork Activity Recognition using Hidden Markov Models
 Team-Oriented Feature Extraction
 Role-Based Teamwork Activity Recognition
 Tuning the Performance of Teamwork Activity Recognition
 Conclusions
Software Tools we Developed
 Teamwork Scenario Editor (TSE)
– Interactive knowledge engineering tool
– Video editor interface
– Visualize large datasets and geographical areas
 All knowledge engineering work in this thesis was
performed using the TSE
 Acquire the datasets
 Find teamwork activities
 Label the activities
Datasets we Acquired, Segmented and
Labeled
 Real-world warfare exercise
 Recorded over a three days
 Data collected from hundreds of soldiers and tanks
equipped with GPS devices, laser range finders and laser
range detectors
Datasets we Acquired, Segmented and
Labeled
 Military Operations in Urban Terrain (MOUT)
Mini-MOUT
OTBSAF
Military Movement
 We extract military movement patterns from the warfare databases
 In the military domain the movement techniques and formations are
selected based on the situational awareness of the team
 Traveling
– Enemy contact not expected
– Fast movement speed
 Traveling overwatch
– Enemy contact possible
– Medium movement speed
– Characterized by continuous movement of lead unit and alternating advancement
of rear units
 Bounding overwatch
– Enemy contact expected
– Slow movement speed
– Alternating or successive bounds
Military Movement
 Formations are used in combination with movement techniques
 Formation is selected based on visibility needs, firepower focus
and so on
 Column
– Leader followed by rear units
– Fire power in all directions (flanks, front and rear)
 Line
– Fire power in the front
 Wedge
– Fire power in the front and in the flanks
 Echelon
– Enemy contact expected in the front or in the echeloned flanks
– Used when one flanks are secured by obstacles
Outline
 Introduction
 Data Acquisition and Knowledge Engineering
 Teamwork Activity Recognition using Hidden Markov Models
 Team-Oriented Feature Extraction
 Role-Based Teamwork Activity Recognition
 Tuning the Performance of Teamwork Activity Recognition
 Conclusions
Teamwork Activity Recognition using
Hidden Markov Models
 A spatio-temporal pattern recognition problem
– Recognize teamwork from X and Y coordinates over time for
multiple agents
 Why Hidden Markov Models?
– Mathematically sound
– Temporal by nature
– Successfully applied in the past (e.g. speech recognition)
 Ways of encoding teamwork activity in HMMs
– Knowledge engineering
– Learn from a set of representative examples
Workflow Overview
Teamwork Activity
Recognition
Teamwork Scenario Editor
External
database
Import
Visualization
Teamwork
Database
Learning
Export
Identification
Representative
Examples
Teamwork Activity
Models
Classification
Real-time
data stream
Annotated
Teamwork
Behavior
The Hidden Markov Model
 The HMM consists of a number hidden states
 Transition probabilities
–
 Emission probabilities
–
– Gaussian PDF
 Initial probabilities
–
HMM with 3 hidden states
The Hidden Markov Model
 Learning algorithms
– Baum-Welch
 Optimize
by maximizing
– Segmental K-means
 Optimize by maximizing
, where is the optimum hidden
sequence given by the Viterbi decoding algorithm
 Classification
– Determine the probability that the input sequence was generated
by the HMM
– Forward evaluation algorithm
Baseline Input Format for the HMM + Basic
Preprocessing
 The input is a vector of the agent positions
– VT = {v1, v2, v3,…, vt}
– vt = {x1, y1, x2, y2, x3, y3, …, xn, yn}
 It is very unlikely that the team action will be repeated in the
same location and position!
 We perform a pre-processing of the input data which allows
us to recognize team actions happening at arbitrary
locations and orientation:
– Translation
 Align with team centroid
– Rotation
 Align with x-axis
 This is not sufficient, many other distorting factors can
happen: scaling, terrain distortion, different ordering of the
agents – we are dealing with these later in this presentation
Experimental Setup
 Test problem
– Real-world military warfare exercise
 Train and test data
– Six activities (extracted using TSE)
– Artificial activities added for testing
Results: Recognition Accuracy
 Classification accuracy is 82%
 Matches the performance of the knowledge engineering
approach
As presented at the AAMAS-07 conference
Results: Real-Time Analysis
 HMM with 4 states and 6 classes  ~9.4ms
Hidden States
Mean Time (ms)
1
2.543
2
4.743
3
7.02
4
9.394
5
12.06
Shortcomings: A Lot of Assumptions!
 We assumed that the teamwork activities were performed at the same
scale
 We assumed that there is no interaction with the environment
 Observation input is of fixed arrangement, hence, we assumed that
recognition is performed on the same team the HMM was trained for
 We assumed that all activities can be modeled using the same number of
hidden states in the HMMs
 Dimensionality and state space for larger teams will quickly grow out of
control
 We do not recognize the roles of the agents in the teamwork activity
– Who is the leader and who is the follower?
 The recognizer is not practical!
Outline
 Introduction
 Data Acquisition and Knowledge Engineering
 Teamwork Activity Recognition using Hidden Markov Models
 Team-Oriented Feature Extraction
 Role-Based Teamwork Activity Recognition
 Tuning the Performance of Teamwork Activity Recognition
 Conclusions
Team-Oriented Feature Extraction
 Improve recognition accuracy and robustness by replacing the “vector of
positions” input with a collection of team-oriented features
– The features are extracted through pre-processing
– Some of the features replace the existing input vector
– Some of the features are the input of the role recognizer (and there is some
overlap)
– It requires a more complex recognition workflow (which will be shown later,
together with the role recognition module)
 Extract semantically rich features from the agent position traces
 Discretization process
– Intuitive descriptions of teamwork activities
– For classifiers using discrete input
 Three feature function classes
– Agent-oriented features
– Environment-oriented features
– Team-oriented features
 Calculated over a sliding window in time
 Features are matched with the ways humans would understand the scene
Agent-Oriented Feature Functions
 Focus on individual agents
 Enhance performance of teamwork recognition
 Used to recognize (likely) roles
Curvature
 The rate at which a curve changes direction
Finite difference approximation
with central difference
Environment-Oriented Feature Functions
 Agent and team interactions with environment
 Environmental objects
– Physical or virtual
– Static or dynamic
 Domain specific examples
– Frontline in war
– Offside line in soccer
– Line-of-scrimmage in football
Angle
Orientation
LOS
A1
Path
t=2
t=1
t=0
Team-Oriented Feature Functions
 Extract features relative to the team
 Specifically designed for teamwork activity recognition
Centroid-Relative Position Vector (CRPV)
 Positions are calculated relative the
centroid position and orientation
 Dimensionality is reduced (compared to
previous approach)
 Translation, rotation and scale invariant
 An evolution is the Role-Relative
Position Vector (RRPV)
– Privileged agent
CRPV Discretization
Cohesion
 Measures the bonding together of the team
– Derived from the Principal Component Analysis
– PCA: Dimensionality is reduced by restricting attention to the
directions along the scatter cloud that are the greatest
 Calculate eigenvalues and eigenvectors from the position-based
scatter matrix
– In the 2D case there are two eigenvalues and two eigenvectors
 Cohesion is the maximum eigenvalue
 CohesionDirection is the direction of the eigenvector with
maximum eigenvalue
 CohesionGradient is the change in cohesion over a sliding
window
Cohesion
 Position-based scatter matrix
Team-Oriented Feature Functions
 Agent-oriented features can be used by replacing the team
with a virtual agent following the team centroid
Outline
 Introduction
 Data Acquisition and Knowledge Engineering
 Teamwork Activity Recognition using Hidden Markov Models
 Team-Oriented Feature Extraction
 Role-Based Teamwork Activity Recognition
 Tuning the Performance of Teamwork Activity Recognition
 Conclusions
Role-Based Teamwork Activity Recognition
 The goal is to improve robustness of the teamwork recognizer
 What are the role assignments in the team?
 The baseline HMM input we presented previously is a fixed
arrangement
– {1, 2, 3, 4} not the same as {2, 3, 4, 1}
 Brute force solution
– Inefficient: One HMM for each permutation
 Role recognition module
– Represent teamwork activity based on roles, rather than by agents
Extended Workflow Overview
Teamwork Scenario Editor
Import
Teamwork
Data Format
Visualization
Identification
Representative
Examples
External
database
Teamwork Activity Recognition Framework
Learning
Export
Teamwork Activity
Models
Role-Oriented Feature
Extraction
Role
Recognition
Role
Models
Recognition
Team-Oriented Feature
Extraction
Assignment
Classification
Mapping
Real-time
data stream
Annotated
Teamwork
Activity
Learning Role Models
 Role models are represented by decision trees
– Trained from observations with ID3 algorithm
– Pruned to minimize effects of overfitting
– Intuitive (white-box)
– Visualizes the exact features which were used in classification
 Follow leader uses CRPV
 Feature functions for role recognition:
– Agent-oriented feature functions
– CRPV feature (assuming that bystander agents are filtered out)
Role Recognition
 Calculate the role assignment probability Pr( ai , rj )
– The probability that agent ai plays role rj
 Extract observation sequences from the movement trace of
each agent
 Input each observation in the sequence to the decision tree
classifier
 The output is a sequence of class frequency vectors
 Role assignment probability:
CRPV
NorthWest
SouthWest
f={2, 4, 0, 0}
f={0, 0, 25, 0}
SouthEast
NorthEast
f={0, 0, 17, 0}
f={4, 6, 0, 0}
South
North
Velocity
f={0, 0, 277, 319}
Low
High
f={66, 113, 0, 0}
Acceleration
Decelerating
f={107, 80, 0, 0}
Constant
f={34, 7, 0, 0}
Accelerating
f={106, 109, 0, 0}
Role Assignment and Mapping
 Identify the best match of role to agent assignments by
searching the role assignment probabilities
– Re-map the team-oriented feature vectors
 Multiple role assignment
– Each agent can play multiple roles
Role Assignment and Mapping
 Unique role assignment
– Each agent can play one role
Experimental Setup
 Warfare exercise dataset
– Same as used previously
– Extended with seven activities
– Four agents and four roles
 Teamwork Activity Recognizer
Activity
Sequences
Observations
Traveling column
29
319
Traveling line
30
330
Traveling box
33
363
Bounding overwatch
9
189
Wedge
17
187
Team split
15
192
Team merge
15
177
– Gaussian Mixture Model (GMM)
– Multiple internal HMMs for each activity to accommodate for complexity variations
in teamwork activities
 GPS readings are not always available (e.g. positions from opponents)
 Simulate noisy observations
– Offset position with a randomly generated number following the Gaussian
distribution multiplied with a noise magnitude
 Mean accuracy and standard deviation was calculated using stratified 10fold cross-validation
Results: Creating the Idealized Team Actions
 Assume perfect role recognition
 Best parameter configuration
– Four hidden states
– Three mixture components
Results: Performance Evaluation with
Unknown Team-Organization
 Shuffled input
Results: Performance Evaluation with
Unknown Team-Organization
 Accuracy with noise and unknown team-organization
– 92.62% with standard deviation 5.53%
– Previous accuracy was 82% (without noise)
 Role recognition improves robustness
 Team-oriented feature functions improves accuracy
Outline
 Introduction
 Data Acquisition and Knowledge Engineering
 Teamwork Activity Recognition using Hidden Markov Models
 Team-Oriented Feature Extraction
 Role-Based Teamwork Activity Recognition
 Tuning the Performance of Teamwork Activity Recognition
 Conclusions
Tuning the Performance of Teamwork
Activity Recognition
 Many components in the recognizer’s workflow are
customizable
– HMM: Number of hidden states and choice of emission
probability representation
– Feature extraction: Width of sliding window and discretization
thresholds
– Role recognition module: Choice of features, parameters in the
ID3 learning algorithm and the choice of learning algorithm
 The probability density function (PDF) has a significant
impact on the accuracy and robustness of the recognizer
Implementation Choice
Multi-modal
Continuous
Parameters
Simple HMM
Yes
No
States (n), Clusters (k)
HMM with Gaussian
No
Yes
States (n)
HMM with GMM
Yes
Yes
States (n), Components (C)
Simple Hidden Markov Model
 Discrete (histogram-based) probability density function
 Inputs are sequences of discrete values
 Vector quantization
– Compress observations
– Training of a codebook
– Cluster assignment
 Advantages
– Multi-modal distribution
 Disadvantages
– Extra cluster parameter to train codebook
– Can not estimate unseen observations
– Requires more training data
Hidden Markov Model with Gaussian PDF
 Used in the initial recognizer
 Advantages
– Continuous
– Can estimate unseen observations
 Disadvantages
– Unimodal distribution
– Example: How to model U-turn to the left and right in the same
HMM?
– Matrix inversion of covariance matrix can be problematic
 Reduce expressiveness by enforcing diagonal covariance matrices
Hidden Markov Model with GMM PDF
 Used in the extended recognizer
 Advantages
– Continuous
– Can estimate unseen observations
– Multi-modal distribution
 Disadvantages
– Extra parameter to determine number of mixture components
– Parameter estimation is computationally expensive
Results: Simple HMM
 Best recognition accuracy
– 60 clusters with 18 hidden
states
– Accuracy is 75.05% ± 14.89%
 Difficulties with
– Bounding overwatch
– Team split
– Team merge
Results: HMM with Gaussian PDF
 Best recognition accuracy
– 3 hidden states
– Accuracy is 91.29% ± 6.69%
 Fails when using more than three
states (not enough training data)
– Bounding overwatch
– Team split
– Team merge
 Reduce expressiveness of the PDF
Results: HMM with GMM PDF
 Best recognition accuracy
– 3 hidden states
– 2 mixture components
– Accuracy is 93.90% ± 4.68%
Summary
 Simple HMM
– Performed poorly
– Filters out too much information
 HMM with Gaussian PDF
– Fails when using the general covariance matrix when n > 3
 HMM with Gaussian Mixture Model PDF
– Best choice
– Least sensitive to changes in parameter configurations
Implementation Choice
Mean
Max
Min
Std. dev.
Simple HMM
62.91%
75.05%
44.52%
5.44%
HMM with Gaussian
83.27%
91.28%
77.76%
4.29%
HMM with GMM
88.30%
93.90%
83.19%
2.17%
Outline
 Introduction
 Data Acquisition and Knowledge Engineering
 Teamwork Activity Recognition using Hidden Markov Models
 Team-Oriented Feature Extraction
 Role-Based Teamwork Activity Recognition
 Tuning the Performance of Teamwork Activity Recognition
 Conclusions
Conclusions

We developed a knowledge engineering tool for visualization,
identification and extraction of teamwork datasets

We developed a teamwork activity recognizer capable of
encoding teamwork activity models from representative
observations

We improved accuracy of the recognizer using team-oriented
feature functions

The robustness of the recognizer was enhanced by integrating
a role-recognition module in the recognition workflow

We studied the importance of probability density function
choices in our recognizer
Contributions
 Teamwork Scenario Editor
 Corpus of labeled teamwork activity
 Team-oriented feature extraction
 Learning teamwork activity from observation
 Role-based teamwork recognition
 Performance tuning of the teamwork recognizer
Future Work
 Team membership assignment
 Larger teams with 100 or more agents
 Preliminary studies
– Kalman filter
 Noise reduction of observation data
 Fusing information from multiple (uncertain) sources
 Applied to estimate life-expectancy in a disaster response simulation
– Genetic programming approach to learn agent strategies
 Applied to a simple foraging game in the single agent domain
Publications
 Teamwork Recognition of Embodied Agents with hidden Markov models. L. J.
Luotsinen, H. Fernlund and L. Bölöni, IEEE 3rd International Conference on Intelligent
Computer Communication and Processing (ICCP07), September 6-8, 2007.
 Automatic Annotation of Team Actions in Observations of Embodied Agents. L. J.
Luotsinen and H. Fernlund and L. Bölöni, Sixth International Conference on Autonomous
Agents and Multiagent Systems (AAMAS-2007), May, 2007, Honolulu, Hawaii.
 A Study of the Robustness of Agent Performance in Nine Popular Agent
Implementation Paradigms. L. J. Luotsinen, M. A. Khan and L. Bölöni, IEEE 3rd
International Conference on Intelligent Computer Communication and Processing (ICCP07),
September 6-8, 2007.
 A Comparison Study of Twelve Paradigms for Developing Embodied Agents. Ladislau
Bölöni, L. J. Luotsinen, Joakim N. Ekblad, T. Ryan Fitz-Gibbon, Charles Houchin, Justin
Key, Majid Ali Khan, Jin Lyu, Johann Nguyen, Rex Oleson, Gary Stein, Scott Vander Welde,
and Viet Trinh. Software: Practice and Experience, 2007.
 Comparing Apples with Oranges: Evaluating Twelve Paradigms of Agency (Book
chapter), L. J. Luotsinen, J. N. Ekblad, T. R. F. Gibbon, C. Houchin, J. Key, M. A. Khan, J.
Lyu, J. Nguyen, R. Oleson, G. Stein, S. V. Weide, V. Trinh and L. Bölöni, ProMAS-2006,
Fourth International Workshop on Programming Multi-Agent Systems, LNAI 4411, pp. 95114, 2007.
 Comparing Apples with Oranges: Evaluating Twelve Paradigms of Agency. L. J.
Luotsinen, Joakim N. Ekblad, T. Ryan Fitz Gibbon, Charles Houchin, Justin Key, Majid Ali
Khan, Jin Lyu, Johann Nguyen, Rex Oleson, Gary Stein, Scott Vander Weide, Viet Trinh
and Ladislau Bölöni, ProMAS-2006, Fourth International Workshop on Programming MultiAgent Systems, Hakodate, Japan, May, 2006.
 A Two-Stage Genetic Programming Approach for Non-Player Characters. L. J.
Luotsinen, Joakim N. Ekblad, Annie S. Wu, Avelino J. Gonzalez and Ladislau Bölöni,
FuturePlay, The International Academic Conference on the Future of Game Design and
Technology, East Lansing, Michigan, October, 2005.
 Collaborative UAV Exploration of Hostile Environments. L. J. Luotsinen, Avelino J.
Gonzalez and Ladislau Bölöni, 24th Army Science Conference, Orlando, Florida, December,
2004.
Publications (Pending)
 Role-Based Teamwork Activity Recognition in Observations of Embodied Agent
Actions. L. J. Luotsinen and L. Bölöni, Submitted to: Seventh International Conference on
Autonomous Agents and Multi-Agent Systems (AAMAS-2008), May, 2008, Estoril, Portugal.
 A Robust Method for Estimating Noisy Measurements Applied to Disaster Response
Operations. L. J. Luotsinen, M. A. Khan and L. Bölöni, 2008.
 Building a World Model under Communication Constraints for Disaster Response
Applications. M. A. Khan, L. J. Luotsinen and L. Bölöni, 2008.
Questions?