ICKEP International Competition for Knowledge Engineering in Planning Lee McCluskey PLANET Knowledge Engineering.

Slides:



Advertisements
Similar presentations
Cultural Heritage in REGional NETworks REGNET Trial Service and Recommendations – D11.
Advertisements

Systems Analysis and Design in a Changing World
Chapter 8: Evaluating Alternatives for Requirements, Environment, and Implementation.
Supporting Business Decisions Expert Systems. Expert system definition Possible working definition of an expert system: –“A computer system with a knowledge.
Session B Wrap-up Summary and Commentary Roman Barták Charles University (Czech Republic)
PLANSERVE – An Intelligent Problem Solving Grid Lee McCluskey and Ron Simpson Artform Research Group, Department of Computing And Mathematical Sciences.
Alternate Software Development Methodologies
Requirements Engineering n Elicit requirements from customer  Information and control needs, product function and behavior, overall product performance,
ICKEP International Competition for Knowledge Engineering in Planning - A PROPOSAL Lee McCluskey KE TCU.
Combining Constraint-based and Classical Formulations for Encoding Planning Domains: GIPO IV Lee McCluskey Artform Research Group, Univ Huddersfield
Chapter 6: Design of Expert Systems
What is Interaction Design?
Knowledge and Systems Research Group, University of Huddersfield B vs OCL: Comparing Specification Languages for Planning Domains Diane Kitchin, Lee McCluskey,
Artificial Intelligence
PLANSERVE Knowledge acquisition & Ontological engineering for AI Planning applications.
Lee McCluskey, University of Huddersfield - EKAW'04 Knowledge Formulation for AI Planning Lee McCluskey Ron Simpson Artform research group Department of.
PDDL: A Language with a Purpose? Lee McCluskey Department of Computing and Mathematical Sciences, The University of Huddersfield.
Usability Inspection n Usability inspection is a generic name for a set of methods based on having evaluators inspect or examine usability-related issues.
Requirements Analysis Concepts & Principles
A Broad View Knowledge Engineering Competition Roman Barták Charles University, Prague What can we.
School of Computing and Mathematics, University of Huddersfield Knowledge Engineering: Issues for the Planning Community Lee McCluskey Department of Computing.
Marakas: Decision Support Systems, 2nd Edition © 2003, Prentice-Hall Chapter Chapter 7: Expert Systems and Artificial Intelligence Decision Support.
School of Computing and Mathematics, University of Huddersfield PDDL and other languages.. Lee McCluskey Department of Computing and Mathematical Sciences,
Informatics Research Group University of Huddersfield Tool Support for Planning and Plan Analysis within Domains embodying Continuous Change Lee McCluskey.
School of Computing and Mathematics, University of Huddersfield Week 21: Knowledge Acquisition / GIPO Lee McCluskey, room 2/09
European Network of Excellence in AI Planning From PLANET to PLANET II Susanne Biundo University of Ulm, Germany.
PLANSERVE – The State of Play – June ‘03 Lee McCluskey The University of Huddersfield, UK.
Introduction to Software Testing
Sepandar Sepehr McMaster University November 2008
Mantova 18/10/2002 "A Roadmap to New Product Development" Supporting Innovation Through The NPD Process and the Creation of Spin-off Companies.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 18 Slide 1 Software Reuse.
Katanosh Morovat.   This concept is a formal approach for identifying the rules that encapsulate the structure, constraint, and control of the operation.
Reuse Standards Dr. Carma McClure Extended Intelligence, Inc. Copyright (c) 1998 by Extended Intelligence, Inc.
Chapter 2 The process Process, Methods, and Tools
Requirements Analysis
Chapter 14: Artificial Intelligence Invitation to Computer Science, C++ Version, Third Edition.
Best Practices By Gabriel Rodriguez
European Network of Excellence in AI Planning PLANET II -- An Overview -- Susanne Biundo University of Ulm, Germany.
European Network of Excellence in AI Planning Knowledge Engineering TCU in PLANET part 2 September, 2001 Lee McCluskey, University.
1 Intelligent Systems ISCRAM 2013 Validating Procedural Knowledge in the Open Virtual Collaboration Environment Gerhard Wickler AIAI, University.
RUP Implementation and Testing
VTT-STUK assessment method for safety evaluation of safety-critical computer based systems - application in BE-SECBS project.
European Network of Excellence in AI Planning Intelligent Planning & Scheduling An Innovative Software Technology Susanne Biundo.
CS 360 Lecture 3.  The software process is a structured set of activities required to develop a software system.  Fundamental Assumption:  Good software.
PLANSERVE - overview of an EU proposal for the “Future and Emerging Technologies” Program Lee McCluskey Artform Research.
1 Chapter 5 Software Engineering Practice. 2 What is “Practice”? Practice is a broad array of concepts, principles, methods, and tools that you must consider.
Module 4: Systems Development Chapter 12: (IS) Project Management.
Jump to first page (c) 1999, A. Lakhotia 1 Software engineering? Arun Lakhotia University of Louisiana at Lafayette Po Box Lafayette, LA 70504, USA.
Technology The practical use of human knowledge to extend human abilities and to satisfy human needs and wan ts.
Understanding and using patterns in software development EEL 6883 Software Engineering Vol. 1 Chapter 4 pp Presenter: Sorosh Olamaei.
Review Meeting, Ulm, Feb 03 Workpackage 1: Synergy in Research and Development Co-ordinators: Malik Ghallab and Lee McCluskey.
Requirements Validation
程建群 博士 (Dr. Jason Cheng) 年 03 月 Software Engineering Part 08.
Computer Science Faculty School of Software Engineering C INTERPRETER AND DEBUGGER (ISO/IEC 9899:2011) Developer: student of 203SE group: Lukyanov Dmitry.
Advanced Software Engineering Lecture 4: Process & Project Metrics.
Welcome to the PRECIS training workshop
1 Learning through Interactive Behavior Specifications Tolga Konik CSLI, Stanford University Douglas Pearson Three Penny Software John Laird University.
ANALYSIS PHASE OF BUSINESS SYSTEM DEVELOPMENT METHODOLOGY.
Types of Information Systems Basic Computer Concepts Types of Information Systems  Knowledge-based system  uses knowledge-based techniques that supports.
1 The Software Development Process ► Systems analysis ► Systems design ► Implementation ► Testing ► Documentation ► Evaluation ► Maintenance.
Software Development Process CS 360 Lecture 3. Software Process The software process is a structured set of activities required to develop a software.
Objective ICT : Internet of Services, Software & Virtualisation FLOSSEvo some preliminary ideas.
Heat Network Demonstration SBRI: policy context & objectives for the competition Natalie Miles Heat Strategy and Policy (Heat Networks)
 System Requirement Specification and System Planning.
GCE Software Systems Development
Lee McCluskey University of Huddersfield
Chapter 6: Design of Expert Systems
Contact person: Mats Brorsson
IS442 Information Systems Engineering
Introduction to Software Testing
Presentation transcript:

ICKEP International Competition for Knowledge Engineering in Planning Lee McCluskey PLANET Knowledge Engineering TCU

Contents Aims / Benefits Current IPC Problems with an ICKEP A small start

Current ICP - Benefits The ICP has brought benefits to the community - - focussed some researchers on technology innovation - led to a rapid development of techniques - delivered a de facto standard for communicating the dynamics of domain models - helped in the validation of planning algorithms and hence led to the sharing of benchmark domain models, tasks and planning tools.

Current ICP - Problems However, the ICP is controversial - it encourages rapid development - but in a narrow area The ICP has assumed that: the input to a planning engine is correct and complete - the language and encoding of a domain into the language are given. the input is in PDDL which was designed to reflect current languages and their underlying assumptions, and with the criterion of “dynamics and nothing else”. It was NOT designed with a model building method in mind OR with many ‘pragmatic’ feature which make building easier - it is more of a machine code than a language for human use!

Narrow views of Planning? Complete, correct, formal, Precondition-effect, Literal strips-based Model of dynamics Something Else? Plan Generator Execution, Scheduling Acquisition, Debugging, Compiling, Configuration, Modelling

Aim of ICKEP The aim of a KE Competition will be to promote the knowledge-based aspects of planning (to include knowledge acquisition, knowledge modelling and domain validation) by evaluating KE tools within a competitive forum. ” knowledge engineering processes support the planning process – they comprise all of the off-line, knowledge-based aspects of planning that are to do with the application being built.”

Possible Benefits it might address the main problem with the current competition - that, although the competition encourages rapid development, it tends to focus work narrowly. it might encourage the development and sharing of stand alone tools to help in the whole process of AI planning including domain modelling, heuristic acquisition, planner-domain matching and so forth. it might lead to some form of communication medium for knowledged-based domain models

Form of Current IPC  Competitors prepare before the event: a planner which can input PDDL and gives out solutions in a prescribed format.  Competitors are given at the event: domain models, tasks, in PDDL  During the event: the planners are executed with the supplied domain models and tasks.  Evaluation after the event: tools are used to rate the planners on speed, coverage, and solution quality.

Form of ICKEP?? But tools and methods to support knowledge acquisition and modelling … do not have standard forms of input. They may acquire knowledge from domain experts or help planning researchers debug domain models. Cannot be easily evaluated by their outputs - what is the advantage of one domain model over another? Are heterogneous - there are several types of tools performing differing functions

PROPOSAL: Start simple Start off with initial competition which has a very simple format, and build from there

Example 1 Competitors prepare before the event: two types of tool (a) one that debugs domain models (b) one that extracts heuristics from domain models. Both tools will input a certain version of PDDL and (a) will output a set of flaws in the domain model, and (b) will output a set of heuristics in a standard format, that can be used with a standard planner, to help solve plan generation problems.

Example 1 Competitors are given at the event: flawed domain models for (a), domain models, a planner and tasks for (b). During the event: the tools are executed with the supplied domain models and tasks. Evaluation after the event: tools are used to rate the competitors' tools for (a) percentage and type of flaws uncovered (b) quality of heuristics acquired as judged by performance improvement on a standard planner.

Example 2 Competitors prepare before the event: software in the form of a standalone tool or tools environment which helps in the process of knowledge engineering for planning. This could be for visualisation, knowledge acquisition, knowledge modelling, domain analysis etc. Planners/Schedulers themselves will NOT be eligible (though they may be part of an environment demonstrated, or they may be used to show the potential of a tool).

Example 2 Evaluation: An evaluation of each submitted software will be made by a Panel, via a demonstration. The Panel will judge the software on criteria with respect to AI Planning/Scheduling, such as the following (by no means exhaustive!): - support potential: what potential has the tool(s) in helping the process of domain acquisition, modelling, visualisation? is the scope of the tool(s) narrow or broad? will the tool make planning software more accessible or usable?

Example 2 Evaluation: - innovation: what is the quality of the scientific and technical innovations that underlie the software? How does it compare with KE software in other areas of AI? - build quality and interoperability: does the software appear robust? can the software be easily used with other planning software, or easily combined with third party planners? Are its interfaces well defined? - relevance: to what degree do the tool(s) address problems peculiar to KE for Planning/Scheduling?

Initial Working Group Prof Ruth Aylet, University of Salford, UK Dr Ronan Bartak, Charles University, Prague, Czech Republic Prof Daniel Borrajo, University Carlos III de Madrid, Spain Prof Susanne Biundo, University of Ulm, Germany Dr Christophe Doniat, Université Technologique de Troyes, France Dr Peter Jarvis, SRI International, USA Prof Lee McCluskey, University of Huddersfield, UK