Transcript Document

Gheorghe Tecuci
Learning Agents Center and Computer Science Department
The Volgenau School of IT & E, George Mason University
[email protected], http://lac.gmu.edu
Apprentissage Artificiel : la Carte, le Territoire et l‘Horizon
Paris, May 23rd 2008
1
Overview
Research Vision: From PC to LA
Disciple Approach to Agents Development
Disciple Principles for Learning Assistants
Applications of Disciple Cognitive Assistants
Final Remarks and Discussion
2
How Did it Start: Yves Kodratoff and Initial Milestones
October 1982: Met Yves at AI
Conference in Bratislava. Encouraged to
do Machine Learning research.
January 1986: First time in France and
the first paper to use the name “Disciple”:
Kodratoff Y. and Tecuci G., "Rule Learning
in Disciple," in Proceedings of the First
European Working Session on Learning,
Orsay, 31 Jan - 2 Feb (Yves: Chair).
July 1988: Thèse de Docteur en Science
Disciple: A Theory, Methodology and
System for Learning Expert Knowledge,
Université de Paris-Sud (Yves: Director)
February 1990: Joined Yves and Ryszard
Michalski at George Mason University.
3
Learning Agents Center: Mission
• Conducts basic and applied research
on the development of knowledge-based
learning and problem solving agents.
Basic
Research
Applied
Research
Tools
Applications
Transitions
• Supports teaching in the areas of
intelligent agents, machine learning,
knowledge acquisition, artificial intelligence
and its applications.
• Develops and applies the Disciple theory,
methodology and agent shells for building
agents that can be taught how to solve
problems by subject matter experts.
Main Collaborators: Mihai Boicu, David Schum, Dorin
Marcu, Vu Le, Cristina Boicu, Marcel Barbulescu, Jerome
Comello, Cindy Ayers, Tom Dybala, Michael Bowman, …
4
Teaching as Alternative to Programming
Building an intelligent machine by
programming is too difficult.
“Instead of trying to produce a
program to simulate the adult
mind, why not rather try to produce
one which simulates the child's?
If this were then subjected to an
appropriate course of education
one would obtain the adult brain.”
Alan Turing
Computing
Machinery and
Intelligence
Mind, 59,
433-460, 1950.
5
How Agents Are Built and Why It is Hard
Traditional Approach: Agent Development by Knowledge Engineer
Subject Matter
Expert
Expert Agent
Knowledge
Engineer
Inference Engine
Dialog
Programming
Results
Knowledge Base
Ed Feigenbaum,
AAAI-93: Rarely
does a technology
arise that offers
such a wide range
of important
benefits. Yet …
Another Approach: Agent Training by Subject Matter Expert
Learning Agent
Subject Matter
Expert
Inference Engine
Dialog
Learning Engine
Knowledge Base
Bill Gates (NYT, 1 March
2004): If you invent a
breakthrough in artificial
intelligence, so machines
can learn, that is worth
10 Microsofts.
6
Disciple Approach to Agent Development
Develop learning and problem solving agents that can be taught by
subject matter experts to become knowledge-based assistants.
The expert
teaches the agent
how to solve
problems in a way
that resembles
how the expert
would teach a
student,
an apprentice or
a collaborator.
The agent
continuously
develops and
refines its
knowledge base to
capture and better
represent expert’s
knowledge and
problem solving
strategies.
There is no longer a clear distinction
between knowledge base development
and its maintenance.
7
Multidisciplinarity and Integration
“The multidisciplinary approach
to problems has become
common practice. However, this
will not be sufficient if we will not
find the integrative factors to
melt the multidisciplinarity into a
unity, be it the case of
understanding a complex reality,
or that of achieving goals
serving the people and society.”
Mihai
Drăgănescu
The Ring of the
Material World,
1989.
8
Multidisciplinarity and Integration in Disciple
Intelligence analysis, Center of gravity determination,
Course of action critiquing, Emergency response
planning, Workaround reasoning, PhD advisor
selection, Teaching higher order thinking skills.
Development of
systematic approach to
expert problem solving
Working closely
with subject
matter experts to
model their
reasoning
DISCIPLE
Army War College
Air War College
George Mason University
Development and
application of
Disciple agents
Disciple
Learning
Agents
Research
Working closely
with end users to
receive crucial
and timely
feedback
Development of the Disciple
theory for agents teaching by
non-computer experts
9
Development of
an agent shell
Lifecycle of a
Disciple agent
Disciple
Knowledge
engineer
1
Agent teaching
by the expert
Disciple
2
Knowledge base integration
and optimization
6
Disciple
Disciple
Disciple
Knowledge engineer and
subject matter expert
Subject matter expert and
knowledge engineer
User teaching
by the agent
3
5
After action review and
agent personalization
Disciple
Disciple
User
User
4
Agent use and
non-disruptive
learning
Users
10
Vision 1: Evolution of Software Development and Use
Mainframe
Computers
Personal
Computers
Learning
Assistants
Software systems
developed and used by
persons who are not
computer experts
Software systems
developed and used
by computer experts
Software systems
developed by computer
experts and used by
persons who are not
computer experts
DISCIPLE
11
Vision 2: Use of Disciple in Education
Disciple
Agent KB
Disciple
Agent KB
The subject matter expert teaches
Disciple similarly to how she or he
would teach a student.
…
teaches
teaches
Disciple
Agent KB
teaches
Disciple teaches the students
similarly to how it was taught.
Personalized Learning: Grand Challenge for the 21st Century
US National Academy of Engineering, February 15th, 2008
12
Overview
Research Vision: From PC to LA
Disciple Approach to Agents Development
Disciple Principles for Learning Assistants
Applications of Disciple Cognitive Assistants
Final Remarks and Discussion
13
Disciple-LTA: Analyst’s Cognitive Assistant
Analytic Assistance
Empowers the analysts through mixed-initiative reasoning for
hypotheses analysis, collaboration with other analysts and experts,
and sharing of information.
Learning
Rapid acquisition
and maintenance
of subject matter
expertise in
intelligence
analysis which
currently
takes years
to establish,
is lost when
experts separate
from service, and
is costly to
replace.
Tutoring
Helps new
intelligence
analysts learn
the reasoning
processes
involved in
making
intelligence
judgments
and solving
intelligence
analysis
problems.
14
Knowledge Base = Object Ontology + Rules
The ontology is
a hierarchical
description of the
domain objects.
Interpretation: Al Qaeda
has chemical and nuclear
weapons as deterrent.
15
Knowledge Base = Ontology + Rules
Rules specify
general
problem reduction
or solution
synthesis steps
Analysis Tree
P1
S1
P11 S11 … P1n S1n
P111 S111 … P11m S11m
Partially
learned
rule
To assess whether
there are states
that may be willing
to sell nuclear
weapons to an
actor, one has to
consider each
nuclear state and
assess whether
that state may be
willing to sell
nuclear weapons
to that actor,
except for the
case in which the
nuclear state is an
enemy of that
actor and also
except for the
case when the
nuclear state
opposes the
proliferation of
nuclear weapons.
16
Disciple Agent Development Approach
Model the
reasoning
of SME
Create
object
ontology
KE
SME
Define
reasoning
rules
Verify and
update rules
Traditionally
With Disciple
Instruct SME
to explicitate
reasoning
KE SME
Develop
reasoning
trees
SME Agent
Import and
develop initial
ontology
KE SME Agent
Specify
Learn
instances ontological
and features elements
SME
Agent
Provide
and explain
solutions
SME Agent
Learn
reasoning
rules
Agent
Analyze
Agent’s
solutions
SME
Explain
errors
SME Agent
Refine
rules
Agent
17
Overview
Research Vision: From PC to LA
Disciple Approach to Agents Development
Disciple Principles for Learning Assistants
Applications of Disciple Cognitive Assistants
Final Remarks and Discussion
18
Disciple Principles for Learning Assistants
1. Problem Solving Paradigm for Expert-Agent Collaboration
2. Multi-Agent and Multi-Domain Problem Solving
3. User Tutoring in Problem Solving
4. Integrated Mixed-Initiative Reasoning
5. Integrated Teaching and Learning
6. Multistrategy Learning
7. Learning within Evolving Representation Space
8. Plausible Reasoning with Partially Learned Knowledge
9. Agent Architecture for Generality-Power Tradeoff
10. Knowledge Base Structuring for Knowledge Reuse
19
P1: Problem Solving Paradigm for Expert-Agent Collaboration
Use a general problem solving paradigm which is:
○ natural for the human user;
○ appropriate for the automated agent.
The reductions and
synthesis operations are
guided by introspective
questions and answers.
Assess whether P
1
Al Qaeda has
nuclear weapons.
It is likely that
Al Qaeda has
nuclear weapons.
S1
P11 S11 … P1n S1n
"I Keep Six Honest..."
I keep six honest serving-men
(They taught me all I knew);
Their names are What and Why and When
And How and Where and Who.
P111 S111 … P11m S11m
Rudyard Kipling
Pa11m Sa11m … Pd11m Sd11m
20
Problem Reduction: Illustration
Main
problems
Abstract
tree
Reduction of a main problem to its main subproblems
Detailed
tree
21
21
Problem Reduction and Solution Synthesis
Detailed evidence
and source analysis
EVD-Dawn-Mir-01-01c
22
Solution Synthesis: Illustration
Analyzed problem
Synthesized solution
This is only an example — not to be
taken as a current analytic estimate
Disciple-LTA makes very clear:
The analysis logic; What evidence was used and how;
What assumptions have been made; What is not known.
23
Assumptions-based Analysis
2. Revised,
assumptionbased, solution
Disciple-LTA allows for: Assumptions checking;
Rapid updating of large analysis trees based on
new intelligence data and assumptions.
Over 1700 reasoning nodes
1. Analyst’s assumption
challenged by Disciple-LTA
24
P2: Multi-Agent and Multi-Domain Hypothesis Analysis
Use a general problem solving paradigm which facilitates:
○ collaboration between users assisted by their agents;
○ solving problems requiring multi-domain expertise.
P1
S1
P1
S1
P11 S11 … P1n S1n
P1n S1n
P111 S111 … P11m S11m
P11 S11
P11m S11m
Pa11m Sa11m … Pd11m Sd11m
25
Service-Oriented Disciple Systems
Problem: Assess
whether Al Qaeda
has nuclear weapons.
Perform
analysis
1
Subproblem: Assess whether other
countries within the global community
believe that Al Qaeda has nuclear weapons.
Subproblem: Assess whether Al Qaeda
makes credible claims to have nuclear weapons.
2
6
Disciple-LTA Client
UDDI
Hypothesis
Analysis
Broker
Check UDDI
for registered
competence
3
Ask broker for solutions
Receive solutions
5
Integrate
Solutions
4
Subproblem: Assess whether other
countries within the global community
believe that Al Qaeda has nuclear weapons.
Solution: It is likely that other countries
within the global community believe that Al
Qaeda has nuclear weapons.
7
Upload
reasoning
tree in
Catalyst
Solicit / receive solutions
Disciple-LTA
Clients
Disciple LTA
Servers
Hypothesis
Analysis
Web Service
Subproblem: Assess whether Al Qaeda makes
credible claims to have nuclear weapons.
Solution: It is almost certain that the Al Qaeda
claims of having nuclear weapons are credible.
P3: User Tutoring in Problem Solving
Abstract
reduction
strategy
Lesson on
Evidence
Automatically
generated
illustration of
the abstract
strategy
27
Lesson Fragment: Hypothesis
support from a piece of evidence
Abstract
synthesis
strategy
Automatically
generated
illustration of
the abstract
strategy
28
P4: Mixed-Initiative Integrated Reasoning
Creative
solution
Context for
creative solution
Modeling
Mixed
Initiative
Reasoning
Expert
example
Problem
Solving
Generated
example
Modeling of Reasoning,
Learning, and
Problem Solving
Problem
Refined
rule
Mixed-Initiative
Problem Solving
Learning
Reasoning Tree
Rule-based
guidance
Ontology + Rules
Extend
Reasoning Tree
Accept
Reasoning Steps
Reject
Reasoning Steps
Learned Rules
Explain
Examples
Explain
Examples
Refined Rules
Refined Ontology
Explain
Examples
Rule Refinement
Rule Learning
29
P5: Integrated Teaching and Learning
Analogy and Hint
Guided Explanation
Analogy-based
Generalization
Rule with Plausible
Version Space Condition
Upper Bound
LowerBound
Example of
problem reduction
step
Incomplete
explanation
analogy
NLP
Knowledge Base
30
Reasoning Rules Learned from Analyst’s Solution
1. The analyst extends
the analysis logic
Learned Rule
2. Disciple learns
reasoning rules
Learned Rule
To assess
whether there
are states that
may be willing to
sell nuclear
weapons to an
actor, one has to
consider each
nuclear state
and assess
whether that
state may be
willing to sell
nuclear weapons
to that actor.
31
P6: Multistrategy Learning
Knowledge Base
Learning by Analogy
and Experimentation
IF we have to solve
<Problem>
Main
PVS Condition
Except-When
PVS Condition
Failure
explanation
Example of problem reductions
generated by the agent
Incorrect
example
Correct
example
Learning from
Explanations Learning from Examples
Except-When
PVS Condition
THEN solve
<Subproblem 1>
…
<Subproblem m>
32
Rules Refined based on Analyst’s Critique
2. The analyst
critiques the
reasoning
1. Disciple applies the learned rule
3. Disciple-LTA refines
the rule with an exceptwhen condition
Refined Rule
This is
wrong!
France will
not sell
nuclear
weapons to
Al Qaeda
because it
perceives it
as an
enemy.
To assess whether there are
states that may be willing to sell
nuclear weapons to an actor, one
has to consider each nuclear
state and assess whether that
state may be willing to sell nuclear
weapons to that actor, except for
the case in which the nuclear state
is an enemy of that actor.
33
Rules Refined based on Analyst’s Critique
2. The analyst
critiques the
reasoning
1. Disciple applies the refined rule
3. Disciple refines the
rule with a new exceptwhen condition
Refined Rule
This is
wrong!
Russia will
not sell
nuclear
weapons to
Al Qaeda
because it
opposes the
proliferation
of nuclear
weapons.
To assess whether there are states that may be
willing to sell nuclear weapons to an actor, one has
to consider each nuclear state and assess
whether that state may be willing to sell nuclear
weapons to that actor, except for the case in which
the nuclear state is an enemy of that actor and also
except for the case when the nuclear state
opposes the proliferation of nuclear weapons.
34
P7: Use of Partially Learned Knowledge
Use confidence level-based reasoning methods that
allow efficient use of partially learned rules for modeling
expert’s reasoning, learning and problem solving.
PVS Condition
Except-When PVS Condition
Rule’s conclusion is
(most likely)
incorrect
Rule’s conclusion is
not plausible
Rule’s conclusion is
(most likely)
incorrect
Rule’s conclusion is
plausible
Rule’s conclusion is
(most likely) correct
35
P8: Learning within Evolving Representation Space
direct
test ev
Continuous
adaptation of
the previously
learned rules to
the evolution of
the ontology.
IF <Problem>
PVS Condition
Except-When
PVS Condition
PVS Condition
Except-When
PVS Condition
THEN <Subproblem 1>
…
<Subproblem m>
36
P9: Architecture for Generality-Power Tradeoff
Structure the architecture of the agent into two parts:
o a reusable domain-independent learning agent shell;
o domain specific modules.
Disciple Agent
Disciple Learning Agent Shell
Graphical User
Interface
Knowledge
Repository
Learner
Problem
Solver
Customized
User Interface
Customized
Problem Solver
Knowledge
Base Manager
Knowledge Base
Domain Independent Modules
Domain Dependent
Modules
37
P10: KB Structuring for Knowledge Reuse
Structure the knowledge base into two parts:
o its more general and reusable components;
o its more specific components.
Knowledge Base
Disciple: Knowledge Base Structuring

The object ontology which may be reused
from existing knowledge repositories;
<object>
Object
Ontology
Scenario
Force
Strategic_COG_relevant_factor
The problem solving rules which are
learned from the subject matter expert.
resource_ or_
infrastructure_element
Other_relevant_factor
Demographic_factor
Civilization_factor
Psychosocial_factor
Economic_factor
Historical_factor
Geographical_factor
International_factor
Political_factor
Military_factor
IF the task is

Force_goal
Strategic_ Operational_
goal
goal
Rules
IF the task is
Identify the strategic COG candidates with respect to the
Identify the strategic COG candidates with respect to the
industrial civilization of a state which is a member of a force
industrial civilization of a state which is a member of a force
IF the task is
IF the task is
The state is ?O2
The state is ?O2
Identify the strategic COG candidates with respect to the
Identify the strategic COG candidates with respect to the
The force is ?O1
The force is ?O1
industrial civilization of a state which is a member of a force
industrial civilization of a state which is a member of a force
IF state
the task
is
IF state
the task
is
explanationThe
explanationThe
is ?O2
is ?O2
Identify
strategic
Identify
strategic
?O2 has_as_industrial_factor
?O3COG candidates with respect to the
?O2 has_as_industrial_factor
?O3COG candidates with respect to the
The
force the
is ?O1
The
force the
is ?O1
industrial civilization
of a state which is a member of a force
industrial civilization
of a state which is a member of a force
?O3 is_a_major_generator_of
?O4
?O3 is_a_major_generator_of
?O4
IF state
the task
is
IF state
the task
is
explanationThe
explanationThe
is ?O2
is ?O2
?O4 IS strategically_essential_goods_or_materiel
?O4 IS strategically_essential_goods_or_materiel
Identify the strategic COG candidates with respect to the
Identify the strategic COG candidates with respect to the
?O2 has_as_industrial_factor
?O2 has_as_industrial_factor
The force is ?O1 ?O3
The force is ?O1 ?O3
industrial
civilization
of a state which is a member of a force
industrial
civilization of a state which is a member of a force
?O3Upper
is_a_major_generator_of
?O4
?O3Upper
is_a_major_generator_of
Plausible
Bound
Condition
Plausible
Bound
Condition
IF state
the task
is
IF the task?O4
is
explanation
The
is
?O2
The state is ?O2
?O4 Force
IS strategically_essential_goods_or_materiel
?O4 Force
ISexplanation
strategically_essential_goods_or_materiel
?O1 IS
?O1 IS
Identify the strategic COG candidates with respect to the
Identify the strategic COG candidates with respect to the
?O2 has_as_industrial_factor
?O2 has_as_industrial_factor
The force is ?O1 ?O3
The force is ?O1 ?O3
?O2 IS Force
Force
industrial
civilization
of a state which is a member of a force ?O2 IS
industrial
civilization
of a state which is a member of a force
?O3Upper
is_a_major_generator_of
?O4
?O3Upper
is_a_major_generator_of
?O4
Plausible
Bound
Condition
Plausible
Bound
Condition
IF?O3
the task
is
IF state
the task
is
explanation
explanation
has_as_industrial_factor
has_as_industrial_factor
The
state
is
?O2
The?O3
is ?O2
?O4
IS
strategically_essential_goods_or_materiel
?O4 Force
IS strategically_essential_goods_or_materiel
?O1 IS Force
?O1 IS
Identify the strategic COG candidates with respect to the ?O3 IS
Identify the strategic COG candidates with respect to the
?O2 has_as_industrial_factor
?O2 has_as_industrial_factor
?O3 IS Industrial_factor
Industrial_factor
The force is ?O1 ?O3
The force is ?O1 ?O3
?O2 IS Force
IS Force
industrial
civilization
of a state which is a member of a force ?O2
industrial
civilization
of a state which is a member of a force
?O3Upper
is_a_major_generator_of
?O4
?O3Upper
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
Plausible
Bound
Condition
Plausible
Bound
Condition
IF state
the task
is
IF state
the task
is
has_as_industrial_factor
has_as_industrial_factor
The?O3
is ?O2
The?O3
is ?O2
?O4 Force
ISexplanation
strategically_essential_goods_or_materiel
?O4 Force
ISexplanation
strategically_essential_goods_or_materiel
?O4 IS Strategically_essential_goods_o_materiel
?O4
IS Strategically_essential_goods_o_materiel
?O1
IS
?O1
IS
Identify
the
strategic
COG
candidates
with
respect
to
the
Identify the strategic COG candidates with respect to the
?O2 has_as_industrial_factor
?O2 has_as_industrial_factor
?O3 IS Industrial_factor
?O3 IS Industrial_factor
The force is ?O1 ?O3
The force is ?O1 ?O3
?O2
IS Force
IS Force
industrial
civilization
of a state which is a member of a force ?O2
industrial
civilization
of a state which is a member of a force
?O3Upper
is_a_major_generator_of
?O4
?O3Upper
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
Plausible
Bound
Condition
Plausible
Bound
Condition
Plausible Lower
Bound
Condition
Plausible Lower
Bound
Condition
IF?O3
the task
is
IF?O3
the task
is
explanation
explanation
has_as_industrial_factor
has_as_industrial_factor
The
state
is ?O2
The
state
is ?O2
?O4 Force
IS strategically_essential_goods_or_materiel
?O4 Force
IS strategically_essential_goods_or_materiel
IS Strategically_essential_goods_o_materiel
?O4
IS Strategically_essential_goods_o_materiel
?O1 IS
?O1 IS
?O1 ?O4
IS Anglo_allies_1943
?O1 to
IS theAnglo_allies_1943
Identify
strategic
Identify
strategic
?O2 has_as_industrial_factor
?O3COG candidates with respect
?O2 has_as_industrial_factor
?O3COG candidates with respect to the
?O3 IS Industrial_factor
?O3 IS Industrial_factor
The
force the
is ?O1
The
force the
is ?O1
IS Force
IS Force
?O2 IS US_1943?O2
?O2 ofISa force
US_1943?O2
industrial
civilization
of a state which is a member
industrial
civilization
of a state which is a member of a force
?O3Upper
is_a_major_generator_of
?O4
?O3Upper
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
Plausible
Bound
Condition
Plausible
Bound
Condition
Plausible Lower
Bound
Condition
Plausible Lower
Bound
Condition
IF state
the task
is
IF state
the task
is
has_as_industrial_factor
has_as_industrial_factor
has_as_industrial_factor
?O3ISexplanation
has_as_industrial_factor
?O3ISexplanation
The?O3
is ?O2
The?O3
is ?O2
?O4
strategically_essential_goods_or_materiel
?O4
strategically_essential_goods_or_materiel
IS Strategically_essential_goods_o_materiel
?O4
IS Strategically_essential_goods_o_materiel
?O1 Industrial_factor
IS
Force
?O1 Industrial_factor
IS
Force
?O1 ?O4
IS Anglo_allies_1943
?O1 to
IS theAnglo_allies_1943
Identify the strategic COG candidates with
Identify the strategic COG candidates with respect to the
?O2 has_as_industrial_factor
?O2 has_as_industrial_factor
?O3 IS
?O3 IS
?O3 IS Industrial_capacity_of_US_1943
?O3 respect
IS Industrial_capacity_of_US_1943
The force is ?O1 ?O3
The force is ?O1 ?O3
?O2
IS
Force
?O2
IS
Force
IF the
task?O4
isof a state which is a member
IF the
task?O4
is of a state which is a member of a force
?O2 IS US_1943is_a_major_generator_of
?O2 ofISa force
US_1943is_a_major_generator_of
industrial
civilization
industrial
civilization
?O3Upper
is_a_major_generator_of
?O3Upper
is_a_major_generator_of
?O4
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
Plausible
Bound
Condition
Plausible
Bound
Condition
Plausible Lower Bound
Condition
Plausible
Lower Bound
Condition
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3ISexplanation
has_as_industrial_factor
?O3ISexplanation
Identify
the?O2
strategic COG candidates with respect
to
the
Identify
the?O2
strategic COG candidates with respect to the
The
state is
The
state is
?O4
strategically_essential_goods_or_materiel
?O4
strategically_essential_goods_or_materiel
?O4
IS Strategically_essential_goods_o_materiel
?O4
IS Strategically_essential_goods_o_materiel
?O4 IS War_materiel_and_transports_of_US_1943
?O4 IS War_materiel_and_transports_of_US_1943
?O1 IS
Force
?O1 IS
Force
?O1 Industrial_capacity_of_US_1943
IS Anglo_allies_1943
?O1 Industrial_capacity_of_US_1943
Anglo_allies_1943
?O2 has_as_industrial_factor
?O2 has_as_industrial_factor
?O3 IS Industrial_factor
?O3 IS Industrial_factor
?O3 IS
IS
industrial
a member
ofISa force
industrial
The
force is civilization
?O1 ?O3of a state which is?O3
The
force is civilization
?O1 ?O3of a state which is a member of a force
?O2 IS Force
?O2 IS Force
?O2
IS
US_1943
?O2
IS
US_1943
?O3
is_a_major_generator_of
?O4
?O3
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
The state is ?O2
The state is ?O2
Plausible
Upper Bound Condition
Plausible
Upper Bound
Condition
THEN
THEN
Plausible Lower Bound
Condition
Plausible Lower Bound
Condition
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3ISexplanation
has_as_industrial_factor
?O3ISexplanation
?O4
strategically_essential_goods_or_materiel
?O4
strategically_essential_goods_or_materiel
ISis aStrategically_essential_goods_o_materiel
IS is aStrategically_essential_goods_o_materiel
The force is ?O1
The force is ?O1
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O4 ISan economic
War_materiel_and_transports_of_US_1943
?O1 IS
Force
?O1 IS
Force
Conclude
COG
COG
?O1 ?O4
ISfactor
Anglo_allies_1943
?O1 ?O4
ISfactor
Anglo_allies_1943
?O2 has_as_industrial_factor ?O3 Conclude that?O3
?O2 has_as_industrial_factor ?O3
?O3 strategic
IS Industrial_factor
?O3 strategic
IS Industrial_factor
?O3 IS Industrial_capacity_of_US_1943
IS Industrial_capacity_of_US_1943
IS Force
IS Force
candidate for a state?O2
whichISis aUS_1943
member?O2
of a force
for a state?O2
whichISis aUS_1943
member?O2
of a force
explanation
explanation
?O3Upper
is_a_major_generator_of
?O4 candidate
?O3Upper
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
THEN
THEN
Plausible
Bound
Condition
Plausible
Bound
Condition
Plausible
Lower
Bound
Condition
Plausible
Lower
Bound
Condition
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
The state is ?O2
state is ?O2
has_as_industrial_factor
?O3IS?O2
has_as_industrial_factor
?O3IS?O2
has_as_industrial_factor
?O3 The
has_as_industrial_factor
?O3
?O4
strategically_essential_goods_or_materiel
?O4
strategically_essential_goods_or_materiel
?O4
ISis aStrategically_essential_goods_o_materiel
?O4
IS is aStrategically_essential_goods_o_materiel
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O1 Industrial_factor
IS
Force
?O1 Industrial_factor
IS
Force
Conclude
COG
Conclude
COG
?O1 Industrial_capacity_of_US_1943
ISfactor
Anglo_allies_1943
?O1 Industrial_capacity_of_US_1943
ISfactor
Anglo_allies_1943
?O3 strategic
IS
?O3 strategic
IS
The force is ?O1?O3 IS
force is ?O1?O3 IS
?O3 is_a_major_generator_of ?O4 The
?O3 is_a_major_generator_of ?O4
IS Force
IS Force
candidate for a state?O2
whichISis aUS_1943
member?O2
of a force
candidate for a state?O2
whichISis aUS_1943
member?O2
of a force
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
The economic
?O3 Plausible Lower
The economic
factor isis_a_major_generator_of
?O3 Plausible Lower
?O4
?O4
THENfactor isis_a_major_generator_of
THEN
Plausible
Bound
Condition
Plausible
Bound
Condition
?O4Upper
IS strategically_essential_goods_or_materiel
?O4Upper
IS strategically_essential_goods_or_materiel
Bound
Condition
Bound
Condition
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
The state is ?O2
The
state
is
?O2
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
ISis aStrategically_essential_goods_o_materiel
IS is aStrategically_essential_goods_o_materiel
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O1 Industrial_factor
IS COG
Force
?O1 Industrial_factor
IS COG
Force
Conclude
Conclude
?O1 ?O4
ISfactor
Anglo_allies_1943
?O1 ?O4
ISfactor
Anglo_allies_1943
?O3 strategic
IS
?O3 strategic
IS
The force is ?O1?O3 IS
The force is ?O1?O3 IS
Industrial_capacity_of_US_1943
Industrial_capacity_of_US_1943
Plausible
Condition
Plausible
Condition
IS
ForceUpper Bound
IS
ForceUpper Bound
candidate for a state?O2
whichISis aUS_1943
member?O2
of a force
candidate for a state?O2
whichISis aUS_1943
member?O2
of a force
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
The economic
?O3 Plausible Lower
The economic
?O3 Plausible Lower
?O4
?O4
THENfactor isis_a_major_generator_of
THENfactor isis_a_major_generator_of
Bound
Condition
Bound
Condition
?O1
IS?O3Force
?O1
IS?O3Force
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
The state is ?O2
The state is ?O2
has_as_industrial_factor
has_as_industrial_factor
?O4
IS
Strategically_essential_goods_o_materiel
?O4
IS
Strategically_essential_goods_o_materiel
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O4thatISan economic
War_materiel_and_transports_of_US_1943
Conclude
is a strategic
Conclude
is a strategic
?O1 ISfactor
Anglo_allies_1943
?O1 ISfactor
Anglo_allies_1943
?O2 Industrial_factor
IS COG
Force
?O2 Industrial_factor
IS COG
Force
?O3
IS
?O3
IS
The force is ?O1?O3 IS
The force is ?O1?O3 IS
Industrial_capacity_of_US_1943
Industrial_capacity_of_US_1943
candidate for a state?O2
whichISis aUS_1943
member of a force
candidate for a state?O2
whichISis aUS_1943
member of a force
has_as_industrial_factor
has_as_industrial_factor
is_a_major_generator_of
?O4 ?O3
is_a_major_generator_of
?O4 ?O3
The economic
factor isis_a_major_generator_of
?O3 Plausible Lower
The economic
factor isis_a_major_generator_of
?O3 Plausible Lower
?O4
?O4
THEN
THEN
Bound
Condition
Bound
Condition
The
state is ?O2
The
state is ?O2
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
?O3
IS Industrial_factor
?O3
IS Industrial_factor
?O4
IS
Strategically_essential_goods_o_materiel
?O4
IS
Strategically_essential_goods_o_materiel
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O4thatISan economic
War_materiel_and_transports_of_US_1943
Conclude
factor
is
a
strategic
COG
Conclude
factor
is
a
strategic
COG
?O1
IS
Anglo_allies_1943
?O1
IS
Anglo_allies_1943
The force is ?O1?O3 IS Industrial_capacity_of_US_1943
The force is ?O1?O3 IS Industrial_capacity_of_US_1943
?O4
?O4
candidate
for a state which
is aUS_1943
memberis_a_major_generator_of
of a force
candidate
for a state which
is aUS_1943
memberis_a_major_generator_of
of a force
The
economic
?O3 IS
The
economic
?O3 IS
is_a_major_generator_of
?O4
is_a_major_generator_of
?O4
THENfactor is?O2
THENfactor is?O2
?O4Lower
IS Bound
Strategically_essential_goods_o_materiel
?O4Lower
IS Bound
Strategically_essential_goods_o_materiel
Plausible
Condition
Plausible
Condition
The state is ?O2
The state is ?O2
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O4thatISan economic
War_materiel_and_transports_of_US_1943
Conclude
is a strategic COG
Conclude
is a strategic COG
?O1 Industrial_capacity_of_US_1943
ISfactor
Anglo_allies_1943
?O1 Industrial_capacity_of_US_1943
ISfactor
Anglo_allies_1943
The force is ?O1?O3 IS
The force is ?O1?O3 IS
candidate for a state?O2
whichIS
is aUS_1943
member
of aBound
force Condition
candidate for a state?O2
whichIS
is aUS_1943
member
of aBound
force Condition
Plausible
Lower
Plausible
Lower
The economic
?O3
The economic
?O3
?O4
?O4
THENfactor isis_a_major_generator_of
THENfactor isis_a_major_generator_of
?O1 IS Anglo_allies_1943
?O1 IS Anglo_allies_1943
The state is ?O2
The state is ?O2
has_as_industrial_factor
?O3
has_as_industrial_factor
?O3
?O4thatISan economic
War_materiel_and_transports_of_US_1943
?O4thatISan economic
War_materiel_and_transports_of_US_1943
Conclude
factor is a strategic COG
Conclude
factor is a strategic COG
?O2 Industrial_capacity_of_US_1943
IS US_1943
?O2 Industrial_capacity_of_US_1943
IS US_1943
The force is ?O1?O3 IS
The force is ?O1?O3 IS
candidate for a state which is a member of a force
candidate for a state which is a member of a force
The economic
?O3 has_as_industrial_factor
The economic
?O3 has_as_industrial_factor
?O4 ?O3
?O4 ?O3
THENfactor isis_a_major_generator_of
THENfactor isis_a_major_generator_of
The state is ?O2
The state is ?O2
?O3
IS Industrial_capacity_of_US_1943
?O3
IS Industrial_capacity_of_US_1943
?O4thatIS
War_materiel_and_transports_of_US_1943
?O4thatIS
War_materiel_and_transports_of_US_1943
Conclude
an economic
factor is a strategic COG
Conclude
an economic
factor is a strategic COG
The force is ?O1
The force is ?O1
?O4
?O4
candidate for a stateis_a_major_generator_of
which is a member of a force
candidate for a stateis_a_major_generator_of
which is a member of a force
The economic
factor is ?O3
The economic
factor is ?O3
THEN
THEN
The
state?O4
is ?O2IS War_materiel_and_transports_of_US_1943
The
state?O4
is ?O2IS War_materiel_and_transports_of_US_1943
Conclude
Conclude
The
force is that
?O1an economic factor is a strategic COG
The
force is that
?O1an economic factor is a strategic COG
THEN
THEN
candidate
forfactor
a stateiswhich
candidate
forfactor
a stateiswhich
The
economic
?O3 is a member of a force
The
economic
?O3 is a member of a force
Conclude
Conclude
The
state is that
?O2 an economic factor is a strategic COG
The
state is that
?O2 an economic factor is a strategic COG
candidate
candidate
The
force is for
?O1a state which is a member of a force
The
force is for
?O1a state which is a member of a force
The statefactor
is ?O2
The statefactor
is ?O2
The economic
is ?O3
The economic
is ?O3
The force is ?O1
The force is ?O1
The economic factor is ?O3
The economic factor is ?O3
38
Overview
Research Vision: From PC to LA
Disciple Approach to Agents Development
Disciple Principles for Learning Assistants
Applications of Disciple Cognitive Assistants
Final Remarks and Discussion
39
Sample Applications of the Disciple Agents
Intelligence Analysis
Emergency Response Planning
Course of Action Critiquing
Workaround Reasoning
PhD Advisor Selection
Higher Order Thinking Skills in History and Statistics
Regulatory KBs for Financial Services Industry
Breast Cancer Treatment Innovations
40
Center of Gravity (COG) of a Force
The center of gravity of an entity (state, alliance, coalition, or
group) is the foundation of capability, the hub of all power and
movement, upon which everything depends, the point against
which all the energies should be directed.
Carl Von Clausewitz, On War, 1832.
The center of gravity of an entity is its primary source of moral
or physical strength, power or resistance.
Joe Strange, Centers of Gravity & Critical Vulnerabilities, 1996.
If a combatant eliminates or influences the enemy’s strategic
center of gravity, then the enemy will lose control of its power
and resources and will eventually fall to defeat. If the
combatant fails to adequately protect his own strategic center
of gravity, he invites disaster. Giles and Galvin, USAWC 1996.
41
The CG-CC-CR-CV Task Reduction Analysis Model
1) Analyzing the strategic COG of a force is
reduced to the tasks of analyzing the COG
candidates corresponding to its main
elements of power (government, people,
economy, military, etc.).
2) Analyzing a COG candidate is reduced to the
tasks of analyzing its Critical Capabilities that
may make it a COG.
3) Analyzing a Critical Capability is reduced to
analyzing its Critical Requirements (i.e. the
essential conditions, resources and means
needed by the critical capability to be fully
operative).
4) Analyzing a Critical Requirement is reduced to
determining whether it has any Critical
Vulnerability (i.e. deficiency, or vulnerability to
neutralization, interdiction or attack in a
manner achieving decisive results)
Analyze
T1
the strategic
COG candidates
for US 1943
T11 S11
Analysis of
President
Roosevelt,
military of US,
industrial capacity,
…
… T S
1n
1n
S1
T111 S111 … T11m S11m
Ta11m Sa11m … Td11m Sd11m
Joe Strange, Centers of Gravity & Critical
Vulnerabilities, 1996 (first printing), 2002
(third printing).
42
Disciple-COG
A learning and decision-making assistant for center
of gravity analysis based on the CG-CC-CR-CV model,
used at the Army War College and the Air War College
43
Disciple-COG at Army War College and Air War College
Center of Gravity Analysis Course
Disciple-COG was taught based on the
expertise of the course’s instructor, who
used the CG-CC-CR-CV model.
Teaching
Learning
Disciple helps military personnel
perform center of gravity analyses
of scenarios of interest.
Disciple
Agent KB
Problem
solving
Sample Evaluations by Officers (Spring 2007)
Tecuci G., Boicu M., Comello J, Agent-Assisted Center of Gravity Analysis, GMU Press, 2008.
44
Disciple-VPT: Collaborating Virtual Experts
for Emergency Response Planning
Emergency situation
Tanker truck leaking red-fuming
nitric acid near a student
residential area of George
Mason University.
Virtual experts needed
•emergency management;
•police operations;
•fire department operations;
•hazardous materials handling;
•health and emergency
medical services;
•sheltering, public works and
facilities;
•federal law enforcement.
VE Library
Disciple-VE
Disciple-VE
Disciple-VE
Disciple-VE
Disciple-VE
Disciple-VE
Disciple-VE
Disciple-VE
Disciple-VE
User
VE Team
VE Assistant
Disciple-VE
Disciple-VE
DISTRIBUTED
KNOWLEDGE BASE
KB
Disciple-VE
KB
KB
Disciple-VE
KB
KB
Disciple-VE
KB
Disciple-VE
45
Disciple-COA: Course of Action Critiquing
Mission:
Close:
Identifies strengths and
weaknesses in a course of
BLUE-BRIGADE2 attacks (BLUE-BRIGADE-OP) to penetrate (BLUE-BRIGADE-TASK) RED-MECH-REGIMENT2 at
130600 Aug in order to enable (ENABLE-MILITARY-PURPOSE1) the completion
of seizebased
(SEIZE2) OBJ-SLAM
by BLUEaction,
on the
ARMOR-BRIGADE1.
principles of war and the
BLUE-TASK-FORCE1, a balanced task force (MAIN-EFFORT1) attacks (ATTACK2) to penetrate (PENETRATE1) REDMECH-COMPANY4, then clears (CLEAR1) RED-TANK-COMPANY2
in order to of
enable
(ENABLE-MILITARYtenets
Army
operations.
PURPOSE2) the completion of seize (SEIZE2) OBJ-SLAM by BLUE-ARMOR-BRIGADE1.
BLUE-TASK-FORCE2, a balanced task force (SUPPORTING-EFFORT1) attacks (ATTACK3) to fix (FIX1) RED-MECHCOMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 in order to prevent (PREVENT-MILITARYPURPOSE1) RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 from interfering with
conducts of the MAIN-EFFORT1, then clears (CLEAR2) RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and
RED-MECH-COMPANY3 and RED-TANK-COMPANY1.
BLUE-MECH-BATTALION1, a mechanized infantry battalion (SUPPORTING-EFFORT2) attacks (ATTACK4) to fix (FIX2)
RED-MECH-COMPANY5 and RED-MECH-COMPANY6 in order to prevent (PREVENT-MILITARY-PURPOSE4) REDMECH-COMPANY5 and RED-MECH-COMPANY6 from interfering with conducts of the MAIN-EFFORT1, then clears
(CLEAR3) RED-MECH-COMPANY5 and RED-MECH-COMPANY6 and RED-TANK-COMPANY3.
Reserve:
The reserve, BLUE-MECH-COMPANY8, a mechanized infantry company, follows Main Effort (MAIN-EFFORT1), and is
prepared to reinforce (REINFORCING-AMILITARY-FORCE1) MAIN-EFFORT1.
Security:
SUPPORTING-EFFORT1 destroys (DESTROY1) RED-CSOP1 prior to begin moving across PL-AMBER by MAINEFFORT1 in order to prevent (PREVENT-MILITARY-PURPOSE3) RED-MECH-REGIMENT2 from observing (MILITARYOBSERVE-ACTION1) MAIN-EFFORT1.
SUPPORTING-EFFORT2 destroys (DESTROY2) RED-CSOP2 prior to begin moving across PL-AMBER by MAINEFFORT1 in order to prevent (PREVENT-MILITARY-PURPOSE6) RED-MECH-REGIMENT2 from observing (MILITARYOBSERVE-ACTION2) MAIN-EFFORT1.
Deep:
Deep operations will destroy (DESTROY3) RED-TANK-COMPANY1 and RED-TANK-COMPANY2 and RED-TANKCOMPANY3.
Rear:
BLUE-MECH-PLT1, a mechanized infantry platoon secures (SECURE1) the brigade support area.
46
DARPA HPKB Program Evaluation
To what extent does COA411 conform
to the Principle of Surprise?
Reference: FM 100-5 pg 2-5, KF 118.1, KF 118.2, KF
118.3 - Surprise is achieved by striking/engaging the
enemy in a time, place or manner for which he is
unprepared. The enemy can be surprised by the
tempo of the operation, the size of the force, the
direction or location of the main effort, and timing.
Factors contributing to surprise include speed,
effective intelligence, deception, application of
unexpected combat power, operations security, and
variations in tactics and methods of operation.
3 GMU
140
120
Performance
Sample Critique: There is a strength
with respect to surprise in COA411
because the enemy is unlikely to be
prepared for the heavy concentration of
combat power applied by BLUE-TASKFORCE1 as MAIN-EFFORT1 in action
PENETRATE1. In this action, MAINEFFORT1 is applying a force ratio of
10.6 which is more than double the
recommended force ratio 3.0. Applying
this much combat power for this action is
likely to surprise the enemy and is
indicative of the proper application of the
principle of surprise."
(Evaluation Items 3, 4, and 5)
160
4
100
4
4
100%
80
5
5
5
ISI
60
TFS/CyCorp
40
20
3
3
Coverage
0
0%
25%
50%
75%
100%
Coverage
• High knowledge acquisition rate.
• Better performance and coverage than
the other evaluated systems.
• Better performance than the evaluating
experts (many unanticipated solutions).
47
Disciple-WA Workaround Agent
Estimates the best plan of working around
damage to a transportation infrastructure,
such as a damaged bridge or road.
25000
Development of Disciple’s KB during evaluation.
72%
of KB
KBsize
sizein 17 days
72%increase
increase of
20000
Evolution of KB coverage and performance from
the pre-repair phase to the post-repair phase.
120
GMU
100
Performance
Disciple-WA showed that a knowledge
engineer can use Disciple to rapidly build
and update a knowledge base capturing
knowledge from military engineering
manuals and a set of sample solutions
provided by a subject matter expert.
ISI
80
60
Tek/Cyc
40
20
Total Axioms
15000
Coverage
0
Rule Axioms
0%
10000
25%
50%
75%
100%
Coverage
Concept Axioms
5000
Task Axioms
Ontology
Tasks
Rules
Knowledge Base
7/
1/
99
6/
17
/9
9
6/
18
/9
9
6/
19
/9
9
6/
20
/9
9
6/
21
/9
9
6/
22
/9
9
6/
23
/9
9
6/
24
/9
9
6/
25
/9
9
6/
26
/9
9
6/
27
/9
9
6/
28
/9
9
6/
29
/9
9
6/
30
/9
9
0
• High knowledge acquisition rate;
• High problem solving performance
(including unanticipated solutions).
• Demonstrated at EFX by Alphatech.
48
Overview
Research Vision: From PC to LA
Disciple Approach to Agents Development
Disciple Principles for Learning Assistants
Applications of Disciple Cognitive Assistants
Final Remarks and Discussion
49
Research Vision for the Disciple Learning Assistants
Learning
Assistants
Personal
Computers
Mainframe
Computers
50
Questions
51
Acknowledgements
The research performed in the Learning Agents Center
was sponsored by several United States government
agencies, including the Air Force Research Laboratory,
the Air Force Office of Scientific Research, the Defense
Advanced Research Projects Agency, the National
Science Foundation, and the Army War College.
52
PVS Condition
Except-When
PVS Condition
Rule with Plaus
Version Space Con
Upper
Lower
53
54
Problem
Mixed-Initiative
Problem Solving
Reasoning Tree
Ontology + Rules
Extend
Reasoning Tree
Accept
Reasoning Steps
Reject
Reasoning Steps
Learned Rules
Explain
Examples
Explain
Examples
Refined Rules
Refined Ontology
Explain
Examples
Rule Refinement
Rule Learning
55