1. http://scom.hud.ac.uk/planet/competition/ ICKEP International Competition for Knowledge Engineering in Planning - A PROPOSAL Lee McCluskey KE TCU

2. Contents Aims / Benefits Current IPC Problems with an ICKEP A small start

3. Current ICP - Benefits The ICP has brought benefits to the community - - focussed some researchers on technology innovation - led to a rapid development of techniques - delivered a de facto standard for communicating the dynamics of domain models - helped in the validation of planning algorithms and hence led to the sharing of benchmark domain models, tasks and planning tools.

4. Current ICP - Problems However, the ICP is controversial - it encourages rapid development - but in a narrow area The ICP assumes that: the input to a planning engine is correct and complete the input is in PDDL which was designed with the criterion of “dynamics and nothing else”. It was designed to reflect current languages and their underlying assumptions. It was NOT designed with a model building method in mind OR with many ‘pragmatic’ feature which make building easier - it is a machine code rather than a language for human use! Plus lots of others I won’t mention

5. Narrow views of Planning? Complete, correct, formal, Precondition-effect, Literal strips-based Model of dynamics Something Else? Plan Generator Execution, Scheduling Acquisition, Debugging, Compiling, Configuration, Modelling

6. Aim of ICKEP The aim of a KE Competition will be to promote the knowledge-based aspects of planning (to include knowledge acquisition, knowledge modelling and domain validation) by evaluating KE tools within a competitive forum.

7. Possible Benefits it might address the main problem with the current competition - that, although the competition encourages rapid development, it tends to focus work narrowly. it might encourage the development and sharing of stand alone tools to help in the whole process of AI planning including domain modelling, heuristic acquisition, planner-domain matching and so forth. it might lead to some form of communication medium for knowledged-based domain models

8. Form of Current IPC for the IPC:  Competitors prepare before the event: a planner which can input PDDL and gives out solutions in a prescribed format.  Competitors are given at the event: domain models, tasks, in PDDL  During the event: the planners are executed with the supplied domain models and tasks.  Evaluation after the event: tools are used to rate the planners on speed, coverage, and solution quality.

9. Form of ICKEP?? But tools and methods to support knowledge acquisition and modelling … do not have standard forms of input. They may acquire knowledge from domain experts or help planning researchers debug domain models. Cannot be easily evaluated by their outputs - what is the advantage of one domain model over another? Are heterogneous - there are several types of tools performing differing functions

10. PROPOSAL: Start simple PROPOSAL: Start off with initial competition which has a very simple format, along the following lines: Competitors prepare before the event: two types of tool (a) one that debugs domain models (b) one that extracts heuristics from domain models. Both tools will input a certain version of PDDL; (a) will output a set of flaws in the domain model, and (b) will output a set of heuristics in a standard format, that can be used with a standard planner, to help solve plan generation problems.

11. PROPOSAL: Start simple Competitors are given at the event: flawed domain models for (a), domain models, a planner and tasks for (b). During the event: the tools are executed with the supplied domain models and tasks. Evaluation after the event: tools are used to rate the competitors' tools for (a) percentage and type of flaws uncovered (b) quality of heuristics acquired as judged by performance improvement on a standard planner.

12. TO DO Is there the will in the community to follow this through? After emailing 12 (?) top US researchers I got ONE reply If we go ahead: Agree on the scope of an initial competition. Make up some outline rules. Target a conference for an initial competition. Get together an organising committee who can also compete!

13. COMPETITION There is a will in the community to follow this through. We will - Agree on the scope of an initial competition. Make up some outline rules. Target a conference for an initial competition. Get together an organising committee who can also compete!

14. Initial Working Group Prof Ruth Aylet, University of Salford, UK Dr Ronan Bartak, Charles University, Prague, Czech Republic Prof Daniel Borrajo, University Carlos III de Madrid, Spain Prof Susanne Biundo, University of Ulm, Germany Dr Christophe Doniat, Université Technologique de Troyes, France Dr Peter Jarvis, SRI International, USA Prof Lee McCluskey, University of Huddersfield, UK