Transcript Modeling Behavioral Activities Related to IED Perpetration

CHALLENGES WITH QUANTIFYING THE
QUALITATIVE
Presented to:
ONR Workshop on Human Interactions in Irregular Warfare as a Complex
System
Atlanta, GA
13-14 April 2011
Presented by:
Dr. Lora Weiss
Georgia Tech Research Institute
[email protected]
In collaboration with:
Elizabeth Whitaker, Erica Briscoe, Ethan Trewhitt, Georgia Tech
Kevin Murphy, Frank Ritter, John Horgan, Penn State
Caroline Kennedy-Pipe, Univ. of Hull
1
LINKING US-UK EXPERTISE
(for understanding IED perpetration in Iraq)
Objective
• Provide a methodology to scientifically capture, evaluate, and predict large-scale behaviors of potential
IED developers before they have successfully deployed devices
• Elicit information from UK subject matter experts, who have had different experiences on their homeland
• Develop analytic tools to conduct quantitative and qualitative analysis of potential interdiction points
SMEs
SMEs
Evaluation
Doctrine
Literature
Knowledge
Engineering
Influence Models
System Dynamics
Models
Agent-based Models
Models
Modeling
Scenarios
Model Considerations
•
•
•
•
Incomplete Data
Data Provenance
Data Uncertainty
Data Perishability
2
Knowledge Engineering Instrument
3
Example Interview Results
Individual
“Western”
Common
Motivations
Common
Goals
Individual
Religion
Goal
Individual
Money
Goal
Individual
Power
Goal
Individual
Activities
Results
Activities
Results
Individual
“Non-Western”
Common End-state vs. Individual Motivations
• Management and planning within IED “teams” are different than in Western civilization
• Participants are not necessarily focused on an end-state. Instead individual motivations
(that may differ) are manipulated toward the individual’s end goals.
4
Interview Results - 2
IRA
Iraqi
Insurgency
Game-like
planning
Little attempt to
Influence monitoring
Counter-IED
Monitoring
Counter-IED
Monitoring
Activities change by being monitored vs. concentrating on technical
execution
• Crucial differences:
– IRA was aware they were being watched and operated in a manner to “fool” their
pursuers
– Iraqi insurgency less of a “game-like” attitude and is more concentrated on purely
technical aspects
5
Interview Results - 3
Current Behavior
Normal
Behavior
Comparison to
spot differences
Unusual
Anomalous
Behavior
Missing
Normal
Behavior
Record
and Share
Stories
Useful information often lost because no explicit sharing of stories
when units transfer
• This information is usually subtle and not directly transcribable, e.g., noticing what
is not normal about an environment (social or physical)
• Military personnel notice things that are different and have a hard time putting
their fingers on exactly what that is
6
Interview Results - 4
Personnel
Recruiting
Religious
Motivation
Monetary
Motivation
Power Motivation
Motivation varies among lower level participants
• For lower level participants (beneath management), motivation is most often
monetary or peer involvement
• Experts are conflicted as to whether religion is actually a motivator or just used as
‘clean’ explanation
7
Mind Map for Preliminary
Knowledge Structuring
8
From Knowledge Engineering to
Modeling
System Dynamics
• Methodology for evaluating of complex systems over time
• Represent causal relationships and feedback
• Stocks and flows represent the movement of items, materials, people, or abstract
concepts
• Easy experimentation with changes in structure, inputs, conditions
9
Simplified Core Model
Central features related to IED perpetration in Iraq
10
Materials and Supplies
• Stock – Materials & Supplies - represents the inventory of generalized
materials and supplies of insurgent groups in the area
• Input Flow - Gathering - represents actions that cause the
accumulation of materials and supplies
• Output Flow - Consumption - represents the use of these materials and
supplies in the construction of IEDs
11
IED Process
• Four stages of IED Process: Constructed, Inventory, Emplaced, Detonated
• Flows between stocks represent transitions from one stage to another
• The Disrupted IED stock and its related flows, Early, Middle and Late
Disruption represent the destruction of IEDs by counter-IED efforts.
12
Personnel
• Represent the transition of a sympathizer into active participation within a terrorist group
• Radicalization represents transition of a person from within the general population to the
Grey Population.
• A previously neutral person taking a position of sympathy for insurgent beliefs
• Deradicalization is the reverse of this, when a person loses sympathy for the insurgency
• As a person becomes an active participant in the IED process, this is represented as
Recruitment
• Death and Disengagement indicate that an active insurgent has left the group in one way or
another.
13
Integrated Model
14
Incorporation of Submodels
15
Personnel
16
Two Radicalization Submodels
Based on method of Bartolomei, J., Casebeer, W.,
& Thomas, T. (2004)
Derived from SME input
17
Representing Culture
Influences
• Complex socio-cultural computational models include both
quantitative and qualitative data
– Qualitative
• Interviews with perpetrators
• Opinions of SMEs
• Broad social, psychological theories
– Quantitative
• Demographics
• Economic Factors
• Surveys
Want to start understanding the interactions of all these influences
 What-If Analyses
18
What-if Analysis
Impact of
Change
Potential
Events
Environmental
Context
Models
Evaluate
Scenarios
Enable analysts to
- Experiment with different sets of parameters, variables, and relationships
- Explore results of
- Events within our control (military actions, policies, diplomatic decisions)
- Events not within our control (weather, crop production, actions of others)
19
Modeling Approaches Across The
Sciences
Figure from: G. Zacharias, J. MacMillan,
and S. Van Hemel (eds), Behavioral
Modeling and Simulation, National
Research Council, 2008.
20
Qualitative Socio-Cultural Data
External Influences
on Behaviors
Opinions, Experiences
of SMEs
Social
Theories
Cultural
Descriptions
Psychological
Theories
Policies
Political Attitudes,
Influences
Interviews with
Individuals or Groups
Being Modeled
Observed
Behavior
21
Quantitative Socio-Cultural Data
Economic Factors
Survey Data
Environmental
Measurements
Demographics
Polls
Census Data
Psychometric
Measurements
22
Representing Qualitative Data in a Computational Model
Representation
Examples
Landmark Values
Left, Right, Straight
Likert Values
Strongly Disagree, Disagree, Neither Agree
nor Disagree, Agree, Strongly Agree
“On a scale of 1 to 10 …”
Fuzzy Values
Low, Medium, High
Relationships and Flows of
Items, Attitudes, Information
Maturity  Employment Stability
Decisions, Rules, Cases
Relational Expressions,
Equations
• Accepted measurement scales may not exist
• Modelers may need to create fuzzy or landmark values for abstract concepts
23
What Kind of Data Does Your Model Need?
• What types of socio-cultural data does
your model need?
• What would you do if you never got it?
• What are realistic substitutes and
workarounds?
24
Realistic Substitutes and Workarounds
Indirect ways to get at the
information, perhaps with a
little more uncertainty
Other types of data that
might be available to
substitute
Creation of data through
laboratory experiments, and
synthetic data created by
software generators
Perhaps a SME’s opinion
included in the model would
provide useful information if
the data remains unavailable
Can we estimate the validity of this kind of
surrogate data for use in a particular model?
25
Dealing with Uncertainty in Socio-Cultural Data
• Where does uncertainty occur?
– Uncertainty in descriptions of attitudes, cultures,
behaviors
– Uncertainty in measurements (physical measurements or
survey instruments)
– Uncertainty in descriptions of historical situations
– Uncertainty inherent in human behavior
• Variations in human choices given the same culture and situation
• What approaches exist for dealing with uncertainty?
– Probabilistic approaches, random variables
– Representations of likelihood (other than strict
probability)
– Techniques for combining certainty values
26
Data Provenance
• Provenance: Where did the data come from, who collected
it, how was it collected, under what circumstances, and
what was the context?
– The model user should understand the provenance of data in order
to determine how appropriate it is for a particular use.
– Data from an authoritative source is not automatically more useful
than data from an unreliable source.
– Data known with a high degree of certainty may not be the data that
leads to recognition of unexpected behaviors.
– Once a piece of information has been confirmed and ‘hits the news’,
it may no longer provide information that can be acted upon.
– In contrast, rumors about conspiracies, although potentially false, are
sources of information that may allow intervention to prevent
catastrophic events.
27
Data Perishability and Missing Data
• Perishability: How long will this data be
valid?
– The importance of knowing when to remove data from a
model and recognizing that behaviors change and adapt
• Model representations in this space need to
be those that can be made robust against
missing features.
28
Summary
Adopt Best Practices for Integrating Qualitative Data into Quantitative Models
Develop
separate
federated
models
Build
iteratively
Build to allow
for easy
extension
Make
limitations
explicit
Different aspects
of the domain
Allows insertion of
new domain
knowledge
New domain
knowledge
Models are built
with simplifying
assumptions
Different views
(micro, meso,
macro)
Provides ability to
change as your
knowledge of the
model changes
Changes in the
system being
modeled
Based on the view
or interpretation
of a modeler
Different time
scales
Allows for
correction and
results in better
models
Changes in the
intended use of
the model
Based on data or
knowledge with
some level of
uncertainty