Transcript Document

Diagrammatic Reasoning in Army
Situation Understanding and
Planning: Architecture for
Decision Support and
Cognitive Modeling
B. Chandrasekaran, Bonny Banerjee,
Unmesh Kurup, John Josephson
The Ohio State University
Robert Winkler
US ARL
Outline of the Talk
• What is Diagrammatic Reasoning? Why is it
important in & for Army Decision-Making?
• Basic research issues & brief outline of progress:
– Representation, Architecture
• Technology built on Science & Applications built
on technology
• Some Remarks on the Future
Ubiquity of Diagrams in
Army Operations
• The Army is about space:
taking it, defending it,
controlling it, avoiding it,
going through it
o Army planning, situation
assessment, situation
monitoring, fusion, all use
diagrammatic representations
o Standards for symbols to be
used are defined in FMs
o
How DR Research Can Help Provide
More Effective Decision Support
• Automating repetitive & routine reasoning tasks that
involve diagrams
– E.g. Critiquing COA’s for vulnerability to ambush
• Better interface design. Understanding what makes a
diagram good, i.e., makes relevant information readily
available, without error, can help in design of decision
interfaces
– Requires how human cognitive architecture & perception work
in performing diagrammatic reasoning
– Cognitive modeling to evaluate diagrammatic interfaces
What Diagrammatic Reasoning is...
• DR is reasoning, i.e. making
inferences and problem solving
with visual representations,
and involves a collaboration
between two systems:
A symbolic reasoning system that
combines information from the
diagram with other information to
make inferences, to set up
diagrammatic perception and
action subgoals
o A diagrammatic representation
from which perception obtains
information about spatial relations
and properties re. diagrammatic
objects
o .
o
The phenomena of
interest can also take
place in our
imagination
Diagrammatic Reasoning is
Not..
• What it is not
o
It is not image processing (such as processing
satellite images for objects of interest, though DR can
be used as part of it)
o
Image processing may be required to extract the diagram
from an image
It is not computes graphics, though diagrams can
often be usefully superimposed on such pictures
o It is not parallel array processing algorithms that solve
problems such as shortest paths, though there is a
role for such algorithms in the overall process of
diagrammatic reasoning,
o
Some Scientific Issues we
Have Made Progress In
• What is a diagram as a representation
– Specificity of a diagram. How are we able to solve a general
problem from a specific diagram?
• Representation in the mind & in the computer.
• The nature of the architecture that can perform
diagrammatic reasoning
– Opportunistic integration of diagrammatic & inferential
operations
• How do diagrams get into long-term memory?
• How are diagrams composed to make new diagrams?
• Abstraction of diagrams
Computational Model of
Diagrammatic Reasoning
• Recall two reasons we mentioned, to
develop computational frameworks for
diagrammatic reasoning:
• Automation or semi-automation.
• Building cognitive models
• Good News!
• A computational architecture that can be
used for automation can also be used for
modeling.
• Our bimodal cognitive architecture: BiSoar
Symbolic Inference &
Perception from Diagrams
• Any system that can support symbolic
representation & inference can be integrates
with our DRS.
Reasoning/
Problem
Solving
System
Perception/
Action
Routines
Diagram
Representatio
n in DRS
• Soar & Act-R happen to be symbolic reasoning
systems with especially useful properties for
general intelligence.
BiSoar: a Bimodal
Cognitive Architecture
• “Thinking” has been usually modeled in AI &
Cognitive Science as syntactic operations on
abstract symbols. Soar, Act-R, etc.
• BiSoar keeps the
general
architecture, but
all states can be
bi-modal; The
agent can have
both linguistic &
pictorial
representations’
Diagrammatic Representation System
• Diagrams consist of three types of objects –
Points, Curves & Regions.
• Diagrams are not just images, they are a spatial
configuration of spatial objects.
The Role of Perception
• Perception and Action Routines: A set of
algorithms that create or modify diagrams and
“perceive” objects and spatial relations between
elements in the diagram.
Emergent Objects & Relations
Reasoning/
Problem Solving
System
Perception/
Action
Routines
Diagram
Representation
in DRS
Perceptual Routines Recognize
C
Emergent Objects and Relations
B
E
ii.
G
A
D
B
B
C
Base set domain-independent, open-ended
J
E
I
i.
A
1.
New object recognition and extraction routines:
A
iii
.
D
H
Intersection-points between line objects, region when a line closes on itself,
new regions when regions intersect, new regions when a line intersects with a
region, extracting distinguished points on a line (such as end points) or in a
region, extracting distinguished segments of a line (such as those created when
two lines intersect), extracting periphery of a region as a closed line. Reverse
operations are included – such as when a line is removed, certain region objects
will no longer exist and need to be removed.
2.
Relational perception routines:
Inside (I1,I2), Outside, Left-of, Right-of, Top-of, Below, Segment-of (Line1,
Line2), Subregion-of (Region1, Region2), On (Point, Line), and Closed (Line1).
3.
Translation, rotation and scanning routines may be combined with
routines in 1 and 2. Example, Intersect (Line, Rotate (90deg, Line
2)).
Action Routines
• Create diagrammatic
objects, such as a path
that goes from point1 to
point2 while avoiding
region2. Path finding and
path modification
routines are especially
useful in Army
applications.
Automatic Synthesis of PR’s & AR’s
• Banerjee’s Ph. D Thesis gives many techniques for
automatic synthesis of PR’s & AR’s. Example: In the
situation below, where c is a wall, A is a member of Red
force, where can BT a member of Blue force hide?
• Once the problem is converted to the
language of geometry, the set of all
points p such that line Ap intersects c, his
techniques can automatically construct
algorithms to solve the problem.
Attention, Learning, & Memory
• BiSoar can be parametrized to mimic the
limitations of human attention & short term
memory.
• BiSoar can learn by a mechanism called
“chunking.” As a result of attention & short term
memory limitations, BiSoar’s LTM contains
smoothed approximations of complex shapes.
Example of Automating DR
• Entity ReIdentification in ASAS “All Source
Analysis System”
• Currently very human-analyst labor intensive,
and many sightings are simply left unattended
Diagrammatic Reasoning in
Information Fusion in an ASAS Problem
• The task is to decide for a newly sighted entity, T3,
which of the previous sighted & identified entities it
is.
Regions impassable
for vehicle types of
interest are marked
and represented
diagrammatically in
the computer
T3
Entities from Past Sightings Retrieved
T2
T1
T3
The Fusion Engine asks for
ways in which T1 & T2 could
have gotten to the location of
T3 within the available time
Two tanks, T1 and
T2 were retrieved
along with their
locations and
times of sighting
Architecture Combines Symbolic &
Diagrammatic Reasoning
For T1 &T2, DR finds eight
possible routes, but rules out
all but one. The figures
shows the routes for T1 & T2.
Example of Action Routine
The Database
reveals that there
are sensor fields
but they didn’t
report any
vehicle
crossings.
A similar question about T2 reveals that T2 also
crossed a sensor field, which also didn’t report any
vehicles. However, DR says T2 could not have
avoided the sensor field.
Numerous Other Applications
•
•
•
•
Rerouting
Ambush vulnerability analysis
Plan critiquing in general
Other uses in information fusion, where the
hypothesis has significant spatial components
Examples of Cognitive Modeling
• Kurup’s thesis models & explores:
– How errors in geographical recall come about.
• Recalled spatial
relationships between
geographical entities
show distortions
• Ex: What is the
relationship between
San-Diego and Reno?
Three Models
• Model 1: Agent has complete map
• Model 2: Agent has symbolic knowledge that SD is South of SF and
Reno is East of SF.
• Model 3: Agent has knowledge that SD in California, Reno in Nevada
and that California is West of Nevada.
C
Reno
N
San-Diego
(a)
(b)
(c)
:(a) Map of SW U.S. in LTM (& WM) of Model
1. (b) & (c) are diagrams in WM constructed by
Models 2 & 3
Models or Route Recall &
Graph Comprehension
• Loss of detail in recall of
routes
– Kurup’s Model posits attention
limits as explanation
• Graph Comprehension
– Lele’s BiSoar model unifies a
variety of observed phenomena
– Using external graphs requires
mental imagistic operations!
DR Automation & Modeling Central
to Decision Support
• The research reported here has laid come scientific
& technological foundations of this area.
• Has also built some demonstration applications &
models.
• But it’s still a baby, there’s potential, but needs to
be nurtured to produce full benefit.
• Many important research issues:
– Extraction of DRS from physical diagrams
– How are appropriate diagrams to help solve problems
generated?