Transcript ulm03 - University of Huddersfield

ICKEP
International Competition for
Knowledge Engineering in
Planning
Lee McCluskey
PLANET Knowledge Engineering TCU
http://scom.hud.ac.uk/planet/competition/
Contents
•
Aims / Benefits
• Current IPC
• Problems with an ICKEP
• A small start
Current ICP - Benefits
The ICP has brought benefits to the community - focussed some researchers on technology innovation
- led to a rapid development of techniques
- delivered a de facto standard for communicating the
dynamics of domain models
- helped in the validation of planning algorithms
and hence led to the sharing of benchmark domain
models, tasks and planning tools.
Current ICP - Problems
However, the ICP is controversial - it encourages rapid
development - but in a narrow area
The ICP has assumed that:
• the input to a planning engine is correct and complete - the
language and encoding of a domain into the language are given.
• the input is in PDDL which was designed to reflect current
languages and their underlying assumptions, and with the
criterion of “dynamics and nothing else”. It was NOT designed
with a model building method in mind OR with many ‘pragmatic’
feature which make building easier - it is more of a machine
code than a language for human use!
Narrow views of Planning?
Acquisition,
Debugging,
Compiling,
Configuration,
Modelling
Complete, correct, formal,
Precondition-effect,
Literal strips-based
Plan
Model of dynamics
Generator
Execution,
Scheduling
Something Else?
Aim of ICKEP
The aim of a KE Competition will be to promote
the knowledge-based aspects of planning (to
include knowledge acquisition, knowledge
modelling and domain validation) by
evaluating KE tools within a competitive
forum.
” knowledge
engineering processes support the
planning process – they comprise all of the
off-line, knowledge-based aspects of planning
that are to do with the application being
built.”
Possible Benefits
• it might address the main problem with the current
competition - that, although the competition
encourages rapid development, it tends to focus work
narrowly.
• it might encourage the development and sharing of
stand alone tools to help in the whole process of AI
planning including domain modelling, heuristic
acquisition, planner-domain matching and so forth.
• it might lead to some form of communication medium
for knowledged-based domain models
Form of Current IPC
 Competitors prepare before the event: a planner
which can input PDDL and gives out solutions in a
prescribed format.
 Competitors are given at the event: domain
models, tasks, in PDDL
 During the event: the planners are executed with
the supplied domain models and tasks.
 Evaluation after the event: tools are used to rate
the planners on speed, coverage, and solution
quality.
Form of ICKEP??
But tools and methods to support knowledge acquisition
and modelling …
• do not have standard forms of input. They may
acquire knowledge from domain experts or help
planning researchers debug domain models.
• Cannot be easily evaluated by their outputs - what is
the advantage of one domain model over another?
• Are heterogneous - there are several types of tools
performing differing functions
PROPOSAL: Start simple
Start off with initial competition which has a very simple
format, and build from there
Example 1
Competitors prepare before the event: two types of
tool
(a) one that debugs domain models
(b) one that extracts heuristics from domain models.
Both tools will input a certain version of PDDL and
(a) will output a set of flaws in the domain model, and
(b) will output a set of heuristics in a standard format,
that can be used with a standard planner, to help
solve plan generation problems.
Example 1
• Competitors are given at the event: flawed domain
models for (a), domain models, a planner and tasks
for (b).
• During the event: the tools are executed with the
supplied domain models and tasks.
• Evaluation after the event: tools are used to rate
the competitors' tools for (a) percentage and type of
flaws uncovered (b) quality of heuristics acquired as
judged by performance improvement on a standard
planner.
Example 2
Competitors prepare before the event: software in
the form of a standalone tool or tools environment
which helps in the process of knowledge engineering
for planning. This could be for visualisation,
knowledge acquisition, knowledge modelling, domain
analysis etc.
Planners/Schedulers themselves will NOT be eligible
(though they may be part of an environment
demonstrated, or they may be used to show the
potential of a tool).
Example 2
Evaluation: An evaluation of each submitted software
will be made by a Panel, via a demonstration. The
Panel will judge the software on criteria with respect
to AI Planning/Scheduling, such as the following (by
no means exhaustive!):
- support potential: what potential has the tool(s) in
helping the process of domain acquisition, modelling,
visualisation? is the scope of the tool(s) narrow or
broad? will the tool make planning software more
accessible or usable?
Example 2
Evaluation:
- innovation: what is the quality of the scientific and
technical innovations that underlie the software? How
does it compare with KE software in other areas of
AI?
- build quality and interoperability: does the software
appear robust? can the software be easily used with
other planning software, or easily combined with third
party planners? Are its interfaces well defined?
- relevance: to what degree do the tool(s) address
problems peculiar to KE for Planning/Scheduling?
Initial Working Group
•
Prof Ruth Aylet, University of Salford, UK
•
Dr Ronan Bartak, Charles University, Prague, Czech Republic
•
Prof Daniel Borrajo, University Carlos III de Madrid, Spain
•
Prof Susanne Biundo, University of Ulm, Germany
•
Dr Christophe Doniat, Université Technologique de Troyes, France
•
Dr Peter Jarvis, SRI International, USA
•
Prof Lee McCluskey, University of Huddersfield, UK