Transcript Computational Discovery of Communicable Knowledge

Symposium on
Reasoning and Learning
in Cognitive Systems
Center for the Study of Language and Information
Stanford University, Stanford, California
March 20-21, 2004
The views contained in these slides are the author’s and do not represent official policies, either
Expressed or implied, of the Defense Advanced Research Projects Agency or the DoD.
Motivation for the Symposium
A number of factors encouraged us to organize this symposium:
 Reasoning and learning are both central aspects of intelligence,
 but the two research groups have become nearly disjoint.
 There exist substantial results on reasoning and learning,
 but many have forgotten or never learned about them.
 There are growing needs for integrated intelligent systems,
 but research focuses primarily on component technologies.
 DARPA now wants cognitive systems that reason and learn.
We hope this meeting can help build a community of researchers
that can respond to these problems and opportunities.
Elements of Machine Learning
performance
element
environment
knowledge
learning
element
Learning to Improve Reasoning
We can state the general task of learning to improve reasoning as:
 Given: Initial knowledge elements for a particular domain;
 Given: A performance system that can compose these elements
dynamically to solve problems or accomplish goals;
 Given: Traces of the performance system’s behavior or advice
about how to solve problems in the domain;
 Find: New or revised knowledge elements that improve system
performance on novel problems.
Much of the early research on machine learning can be cast in just
these terms.
Some Systems that Reason and Learn
STRIPS (1972)
Anzai (1978)
ACT-F (1981)
LEX (1981)
SAGE (1982)
UPL (1983)
Soar (1984)
MORRIS (1985)
LEAP (1985)
MacLearn (1985)
Prodigy/E (1988)
Eureka (1989)
Bagger (1990)
PRIAR (1990)
Daedalus (1991)
Cascade (1993)
Prodigy/A (1993)
SCOPE (1996)
Characteristics of Early Research
1. The performance system engaged in multi-step reasoning by
dynamic composition of knowledge elements.
2. Learning methods were typically incremental and integrated
with the performance system.
3. Learning was relatively rapid and took at least some domain
knowledge into account.
4. Learning was embedded in a problem-solving architecture that
made representational and performance assumptions.
5. Research emphasized support of cognitive abilities, such as
planning and reasoning, rather than perception and execution.
6. Researchers looked to psychology and logic for ideas, rather
than to statistics and operations research.
Some Historical Developments
1959
1972
1978
1980
1981
1983
1986
1988
1989
1991
1992
1993
1995
1998
Creation of the General Problem Solver
Development of STRIPS with MACROPs
First adaptive production systems developed
Carnegie symposium on learning and cognition
Growth of work on learning in problem solving
Active research on cognitive architectures
Growth of explanation-based learning movement
Recognition of the utility problem
Rise of experimental method, advent of UCI repository
ISLE/Stanford symposium on learning and planning
Influx of ideas from pattern recognition
Excitement about reinforcement learning
Influx of ideas from operations research
Reduced effort on learning and reasoning
Some Encouraging Signs
In recent years, there have been some positive developments:
 academic courses and tutorials on learning and reasoning;
 AI Magazine survey of work on learning in planning domains;
 interest in model-based and relational reinforcement learning;
 broader interest in integrated cognitive architectures;
 DARPA workshop on rapid, embedded, and enduring learning;
 prospects for DARPA program in learning for cognitive systems.
Taken together, these suggested the time had arrived for another
meeting on reasoning and learning.
Some Omitted Paradigms
The meeting has some great speakers reporting on great topics, but
some may wonder why there are no talks on:
 probabilistic learning and reasoning in Bayesian networks;
 model-based approaches to learning from delayed reward;
 learning action models for use in planning and execution.
Each framework can learn knowledge that supports some form of
multi-step reasoning or inference.
However, research in these paradigms focuses on statistical issues
rather than structural ones, which we emphasize here.
Some Open Research Problems
Previous research in the area of learning and reasoning has:
 focused on acquisition of relatively small knowledge bases;
 dealt with learning over relatively short periods of time;
 emphasized mental processes over action and perception;
 preferred logical, all-or-none frameworks over alternatives;
 downplayed the role of hierarchical knowledge structures;
 relied primarily on initial, handcrafted representations.
Each of these suggests open problems that should be addressed
in future projects.
Challenge: Learning to Improve Reasoning
Current learning research focuses on performance tasks that:
 involve one-step decisions for classification or regression;
 utilize simple reactive control for acting in the world.
But many other varieties of learning instead involve:
 the acquisition of modular knowledge elements that
 can be composed dynamically by multi-step reasoning.
We should give more attention to learning such compositional knowledge.
knowledge
reasoning
knowledge
reasoning
Challenge: More Rapid Learning
Current learning research focuses on asymptotic behavior:
 methods for learning classifiers from thousands of cases;
 methods that converge on optimal controllers in the limit.
In contrast, humans are typically able to:
 learn reasonable behavior from relatively few cases;
 take advantage of knowledge to speed the learning process.
performance
We need more work on knowledge-guided learning of this variety.
experience
Challenge: Cumulative Learning
Current learning research focuses on isolated induction tasks that:
 take no advantage of what has been learned before;
 provide no benefits for what is learned afterwards.
In contrast, much human learning involves:
 incremental acquisition of knowledge over time that
 builds on knowledge acquired during earlier episodes.
We need much more research on such cumulative learning.
initial knowledge
extended knowledge
Challenge: Evaluating Embedded Learning
Current evaluation emphasizes static data sets for isolated tasks that:
 favor work on minor refinements of existing component algorithms;
 encourage mindless “bake offs” that provide little understanding.
To support the evaluation of embedded learning systems, we need:
 a set of challenging environments that exercise learning and reasoning,
 that include performance tasks of graded complexity and difficulty, and
 that have real-world relevance but allow systematic experimental control.
battle management
in-city driving
air reconnaissance
An Advertisement: Progress on ICARUS
We are extending ICARUS, an integrated cognitive architecture that:
 stores long-term knowledge as hierarchical skills and concepts;
 encodes short-term elements as instances of long-term structures;
 uses numeric value functions to select skill paths for execution;
 modulates reactive behavior with a bias toward persistence;
 learns value functions for concepts and durations of skills;
 invokes means-ends analysis to handle unexecutable skills;
 learns new hierarchical skills upon resolution of impasses;
Come to our poster this evening to hear more about the system.
End of Presentation