1. G OAL -O RIENTED C ONCEPTUALIZATION OF P ROCEDURAL K NOWLEDGE Martin Možina, Matej Guid, Aleksander Sadikov, Vida Groznik, Ivan Bratko Artificial Intelligence Laboratory Faculty of Computer and Information Science University of Ljubljana, Slovenia ITS 2012

2. C ONCEPTUALIZATION OF P ROCEDURAL K NOWLEDGE ORIGINAL THEORYPROBLEM SOLUTION....................................................................... axioms laws formulas rules of the game … path: requires excessive computation, difficult to memorize CONCEPTUALIZED DOMAIN THEORY DECLARATIVE KNOWLEDGEPROCEDURAL KNOWLEDGE basic domain knowledgegoal-oriented rules

3. C ONCEPTUALIZATION OF P ROCEDURAL K NOWLEDGE ORIGINAL THEORYPROBLEM SOLUTION....................................................................... axioms laws formulas rules of the game … path: requires excessive computation, difficult to memorize CONCEPTUALIZED DOMAIN THEORY DECLARATIVE KNOWLEDGEPROCEDURAL KNOWLEDGE basic domain knowledgegoal-oriented rules

4. C ONCEPTUALIZATION OF P ROCEDURAL K NOWLEDGE ORIGINAL THEORYPROBLEM SOLUTION....................................................................... axioms laws formulas rules of the game … path: requires excessive computation, difficult to memorize CONCEPTUALIZED DOMAIN THEORY basic rules of chess piece movements the 50-move rule … the “right” corner concept basic strategy … procedures: IF-THEN rules simple and compact rules easy to memorize … intuitive knowledge intermediate goals … DECLARATIVE KNOWLEDGEPROCEDURAL KNOWLEDGE

5. P ROBLEM S TATE S PACE :::: :::: ::::... start node goal nodes (too) long solution path...

6. L EARNING I NTERMEDIATE G OALS :::: :::: ::::... goal nodes of intermediate goals start nodes of intermediate goals...

7. K NOWLEDGE E LICITATION WITH ABML IF... THEN...... ABML argument-based machine learning experts’ arguments constrain learning obtained models are consistent with expert knowledge Možina M. et al. Fighting Knowledge Acquisition Bottleneck with Argument Based Machine Learning. ECAI 2008. arguments critical examples counter examples experts introduce new concepts (attributes) human-understandable models (suitable for teaching)

8. B ENEFITS OF ABML FOR K NOWLEDGE E LICITATION IF... THEN...... ABML argument-based machine learning explain single example easier for experts to articulate knowledge “critical” examples expert provides only relevant knowledge “counter” examples detect deficiencies in explanations Možina M. et al. Fighting Knowledge Acquisition Bottleneck with Argument Based Machine Learning. ECAI 2008. arguments critical examples counter examples

9. G OAL -O RIENTED R ULE L EARNING GOAL EVALUATION: is the goal achievable? does the goal always lead to progress? G OAL -O RIENTED R ULE L EARNING

10. Computer (to the expert): “What goal would you suggest for white in this position? What are the reasons for this goal to apply in this position?” G OAL -O RIENTED R ULE L EARNING : A “C RITICAL ” E XAMPLE

11. Computer (to the expert): “What goal would you suggest for white in this position? What are the reasons for this goal to apply in this position?” G OAL -O RIENTED R ULE L EARNING : A “C RITICAL ” E XAMPLE The expert (a FIDE master): “White can squeeze black king’s area. It is possible to build a barrier and squeeze the area available to the black king.”

12. Computer (to the expert): “What goal would you suggest for white in this position? What are the reasons for this goal to apply in this position?” G OAL -O RIENTED R ULE L EARNING : A “C RITICAL ” E XAMPLE The expert (a FIDE master): “White can squeeze black king’s area. It is possible to build a barrier and squeeze the area available to the black king.”

13. G OAL -O RIENTED R ULE L EARNING : A “C OUNTER ” E XAMPLE Computer found an example where current goal “squeeze black king's area” does not lead to progress. 1.Kf5-g5: mate in 8 moves (optimal execution) 1.Bg4-e2: mate in 10 moves (worst execution) In this case, the expert found this execution of the goal to be perfectly acceptable. HUMAN-UNDERSTANDABLE PLAY X OPTIMAL PLAY no progress…. Computer: “Would you admonish a student if he or she played 1.Bg4-e2 in this position?”

14. C ONCEPTUALIZATION OF D OMAIN K NOWLEDGE : C HESS E NDGAME goal-oriented instructions example games with goal-oriented instructions KBNK – the most difficult of elementary chess endgames: several recorded cases when even grandmasters failed to win the result of conceptualization: Hierarchy of (only) 11 GOALS

15. T EACHING M ATERIALS (1): T EXTBOOK I NSTRUCTIONS

16. T EACHING M ATERIALS (2): E XAMPLE G AMES WITH G OALS

17. A G RANDMASTER F AILED TO W IN... A grandmaster of chess failed to win the following endgame…

18. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

19. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

20. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

21. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

22. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

23. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

24. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

25. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

26. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

27. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

28. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

29. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

30. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

31. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

32. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

33. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

34. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

35. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

36. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

37. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

38. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

39. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

40. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

41. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

42. A G RANDMASTER F AILED TO W IN... GM Kempinski (white) – GM Epishin (black), Bundesliga 2001

43. A G RANDMASTER F AILED TO W IN... … but why our students didn’t?

44. I NTERMEDIATE G OAL : B UILD A B ARRIER

52. R ESULTS OF A P ILOT E XPERIMENT PARTICIPANTS three chess beginners of different strengths TEACHING MATERIALS goal-oriented textbook instructions example games with instructions Phase II: Examination of teaching materials participants were given access to teaching materials Phase III: Playouts against optimally defending computer after each game (but not during the games!) the materials were accessible to participants The students learned the skill operationally in up to an hour’s time of studying the instructions and testing their skill in actual problem solving (playing the endgame). Phase I: Three trial KBNK games participants were unable to deliver checkmate

53. G OAL -O RIENTED C ONCEPTUALIZATION OF P ROCEDURAL K NOWLEDGE human assimilable simple and compact easily executable ABML: powerful knowledge elicitation method next step: ITS for teaching chess endgames learning intemediate goals +

54. http://www.ailab.si/matej/ dr. Matej Guid. Artificial Intelligence Laboratory, Faculty of Computer and Information Science, University of Ljubljana. Research Page: www.ailab.si/matej