Transcript Knowlede Engineering: Prosecute ASW Contact DPM

COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
ARO Workshop on
Cyber Situation Awareness
RPD-inspired Hypothesis Reasoning for
Cyber Situation Awareness
November 14, 2007
John Yen, Mike McNeese, and Peng Liu
NCSD-ADS-DOC-3810-2.0-20070412
Wagner Associates
Overview
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
• Cognitive Foundation: RPD Model
• RPD-enabled Collaborative Agents: RCAST
• Hypothesis Reasoning in R-CAST
• Similarity-based Activation of
Hypothesis
• Gathering Missing Relevant Information
2
Recognition-Primed Decision
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
• A cognitive model of human decision-making
under time pressure.
• A naturalistic decision-making model
• A holistic decision-making model
– Includes gathering relevant information
– Captures the entire decision making process,
not just the “decision point”.
• An adaptive decision-making process
– Includes detecting changes in the environment
so that decisions can be adapted.
3
Three Types of Relevant Information
in RPD Model
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
start
Situation analysis
– Missing Cues
– Criteria for Evaluating
Options
Investigation
anomaliesdetected
Feature matching
miss
information
complete
information
– Expectancy
Expectancy monitor
Evaluate option
not workable
workable
Implement
option
Learning
end
Adapted from G.A. Klein 1989
4
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
RPD-enabled Agents: R-CAST
Investigation in RPD
start
Situation analysis
Investigation
anomalies detected
Feature matching
miss
information
Expectancy monitor
Information Manager in R-CAST
complete
information
R-CAST
Evaluate COA
not workable
Anticipate Information Requirements
workable
Implement
COA
Experiences
Evaluation
Criteria
Plan
Knowledge
Deliberated decisions:
What to do?
How to evaluation options?
How to implement it?
Learning
RPD Model
end
RPD Decision
Model
Decisions
Recommender
Who needs it?
Deadline?
What cues are needed?
Knowledge base
New/missing
information
Information
manager
Option
Process
manager
Execute/Monitor
What expectancies are monitored?
Information
Requirements
Communication
manager
Relate high-level info needs
to lower-level information
How to seek/share information?
How to communicate?
Inference Rules
Investigation
Strategies
Directory &
protocol
Conversations
Manage Information Requirements
5
Hypothesis Reasoning
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
• Hypothesis guides the seeking of relevant
information.
supporting
Sub1
Sub2
Sub2
Sub3
Sub1
Sub3
Sub2
Option
1
Option
2
Sub3
Option
2
Option
1
Page 6
RCAST
Sub4
Sub1
Page 1
Sub1
Sub2
evolving
Sub2
Option
1
Option
1
Sub1
Page 5
Sub3
RCAST
Page 2
Home Page
Sub1
Option
2
Sub2
Option
2
Sub2
Sub3
Page 4
Sub1
Page 3
Option
1
Sub1
Option
1
Sub2
Sub2
Option
2
Sub3
Option
2
RCAST
O
Sub2
Sub1
Sub1
Sub2
forming/refining
Collaborative
Evidence
Chaining
O
Hypothesis Space
Evidence Space
Sub1
Sub1
Sub2
Collaborative
Decision
Making
triggering
6
Hypothesis Reasoning in R-CAST
Goal /
Situation
Decision
Manager
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
Action
Information
Manager
Knowledge-base
Manager
Knowledge
Base
Communication
Manager
Agent
Directory
Hypothesis
Manager
Multi-Layer
Bayesian
Network
7
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
8
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
Similarity-based Activation of Hypotheses
• Based on similarity-based
matching with cues of
“Experience”
• Allows for partial matching
• Cues can be associated
with weights
• Variable bindings of
hypotheses are established
by the matching process.
Experience e1
Cue:
C1
C3
C5
Hypothesize B
9
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
Similarity-based Matching for Hypothesis
Activation
Hypothesis Type D
e12
e5
e6
e14
Hypothesis Type A
Hypothesis Type C
e10
e7
e9
e4
e3
Recommended
Hypothesis
X
e8
e1
Hypothesis Type B
e2
Closest Experiences
For Alternative Hypotheses
Current Situation
10
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
Hypothesis Activation
Experience
C1
C2
C3
C4
C5
Hypothesis
e1
Large
-
Yes
-
?
B
-
-
-
-
A
e3
e8
-
-
Violated
-
C
e14
-
-
-
-
D
• Shows the hypothesis that matches the current situation best
• Presents option analysis for alternative hypotheses
Matching cues of the recommended hypothesis
Matching cues of alternative hypothesis
Cues not applicable for a hypothesis
Unknown cues relevant for a hypothesis
11
Option Analysis for Alternative
Hypotheses
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
C1
C2
C3
C4
C5
Hypothesis
Large
-
Yes
-
?
B
-
No
-
-
A
-
-
Violated
-
C
-
-
-
-
D
• Shows what conditions would have resulted in alternative
hypothese
• Blue cells indicate conditions identical to the current situations
• Example:
– If C3 did not occur,
the recommended hypothesis would have been A
12
Overview
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
• Cognitive Foundation: RPD Model
• RPD-enabled Collaborative Agents: RCAST
• Hypothesis Reasoning in R-CAST
• Similarity-based Activation of
Hypothesis
 Gathering Missing Relevant Information
• Automated Update/Refine of Hypothesis
13
COLLEGE OF INFORMATION
R-CAST Automates Gathering
SCIENCES AND TECHNOLOGY
Relevant Information
Four sources of information for matching with experiences
1.
2.
3.
4.
Facts in knowledge base
Inference rules in knowledge base
External services
Hypothesis
Hypothesis
Manager
Experience
C5?
Cues
C1
C3
C5
RPD Decision
Model
Information
Manager
C3 ?
Hypothesis
Communication
Manager
C9 ?
C9
Service
Knowledge
Base
B
Facts
Inference Rules
C9
C3
C1
14
Gather Missing Information
Through Backward Reasoning and
Hypothesis
Information
Manager
RPD Decision
Model
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
Experience
Cues
C3
Inference Rules
Information
Requirement
Decision
F
Missing
Information
Hypothesize B
D
C3
G
Known
E
Missing
Information
H
Known
Request: E
Hypothesize F
Agent
15
Summary
COLLEGE OF INFORMATION
SCIENCES AND TECHNOLOGY
• RPD-based agents enable similarity-based activation of
hypotheses
– Allow for incomplete information
– Enable comparison with alternative hypotheses
• Reasoning about missing relevant information
– Through backward inference
• Potential for Cyber Situation Awareness
– Using hypothesis reasoning to infer missing information
– Using hypothesis reasoning to reduce false positive alerts.
Current Efforts
• A novel integration of Bayes Net with predicate logic for
missing information reasoning.
• Refinement of hypotheses through reasoning about their
variable bindings.
16