1. The 6 th Computer Olympiad: Computer-Games Workshop (Maastricht, 2001) Advances of AND/OR-tree Search Algorithms in Shogi Mating Search Hiroyuki Iida and Makoto Sakuta Department of Computer Science Shizuoka University, Japan

2. Overview Tsume-shogi (mating search) problem is a good application for AND/OR-tree search algorithmsTsume-shogi (mating search) problem is a good application for AND/OR-tree search algorithms Algorithms for solving tsume-shogiAlgorithms for solving tsume-shogi –ID, AO*, PN search, PN*, PDS and DF-PN –Equivalent relations between these algorithms Tsume-shogi in actual shogi endgameTsume-shogi in actual shogi endgame –Find a relatively long-step winning sequence

3. Main rules of tsume-shogi Two agents: attacker and defenderTwo agents: attacker and defender Attacker has to continue the check move sequences.Attacker has to continue the check move sequences. Attacker must mate in the shortest sequence.Attacker must mate in the shortest sequence. Defender must select a move which makes the mating sequence as long as possible.Defender must select a move which makes the mating sequence as long as possible. The length of the solution sequence is not given in advance.The length of the solution sequence is not given in advance. See also [Seo, Iida and Uiterwijk, 2001]See also [Seo, Iida and Uiterwijk, 2001]

4. Why tsume-shogi is a good application for AND/OR-tree search? Tsume-shogi has an enormous search space, so simple brute-force search is not adequate.Tsume-shogi has an enormous search space, so simple brute-force search is not adequate. A tsume-shogi problem has a unique solution.A tsume-shogi problem has a unique solution. There are various types of problems, including those very hard to solve and those having a very long solution sequence.There are various types of problems, including those very hard to solve and those having a very long solution sequence. Many high-performance tsume-shogi solving programs have been developed, so the performance of a program can be measured and compared.Many high-performance tsume-shogi solving programs have been developed, so the performance of a program can be measured and compared.

5. Examples of solving tsume-shogi (1) An endgame position from GM’s game. (mate in 17 plies, relatively not so long)An endgame position from GM’s game. (mate in 17 plies, relatively not so long) –ID solved in 22.6 seconds. –PN solved in 7.93 seconds. –PDS solved in 6.11 seconds. –PN* solved in 4.87 seconds. See the detail (machine spec, size of transposition table, etc.) in [Sakuta and Iida, ACG9].See the detail (machine spec, size of transposition table, etc.) in [Sakuta and Iida, ACG9].

6. Examples of solving tsume-shogi (2) An endgame position from GM’s game. (mate in 37 plies, relatively long in actual games)An endgame position from GM’s game. (mate in 37 plies, relatively long in actual games) –ID was unable to solve. –PN was unable to solve. –PN* solved in 1000 seconds. –PDS solved in 757 seconds. See the detail (machine spec, size of transposition table, etc.) in [Sakuta and Iida, ACG9].See the detail (machine spec, size of transposition table, etc.) in [Sakuta and Iida, ACG9].

7. Why proof-number search? Best-first search algorithms such as proof-number (PN) search work well when solving endgame positions in chess-like games. (also other games: GO, LOA, etc.)Best-first search algorithms such as proof-number (PN) search work well when solving endgame positions in chess-like games. (also other games: GO, LOA, etc.) However, it is hard, even for PN search, to solve an endgame problem with very long sequences because of its inherent memory limits.However, it is hard, even for PN search, to solve an endgame problem with very long sequences because of its inherent memory limits. Recently, innovative depth-first search algorithms based on proof/disproof numbers were developed (PN*, PDS and DF-PN) for solving tsume-shogi problems, among which one with 1525 steps.Recently, innovative depth-first search algorithms based on proof/disproof numbers were developed (PN*, PDS and DF-PN) for solving tsume-shogi problems, among which one with 1525 steps. These depth-first algorithms behave like best-first search. These depth-first algorithms behave like best-first search.

8. AND/OR-tree search Best-first searchBest-first search AO* (Nilsson) and its variants Minimax tree CN-search (McAllester, 1988) (Schaeffer, 1990) conspiracy number PN-search (Allis,1994) proof number disproof number A* for OR Graph

9. PN-search (Allis, 1994) PN (proof number) search is a best-first AND/OR tree-search algorithm.PN (proof number) search is a best-first AND/OR tree-search algorithm. Inspired by conspiracy-number search algorithm (McAllester, 1988).Inspired by conspiracy-number search algorithm (McAllester, 1988). Before starting the search, a search goal is defined. E.g., try to mate. The evaluation of a node returns one of three values: true, false, or unknown. The evaluation is seen from the point of view of the player to move in the root position. The value true indicates that the player to move in the root position can achieve the goal, while false indicates that the goal is unreachable.Before starting the search, a search goal is defined. E.g., try to mate. The evaluation of a node returns one of three values: true, false, or unknown. The evaluation is seen from the point of view of the player to move in the root position. The value true indicates that the player to move in the root position can achieve the goal, while false indicates that the goal is unreachable. A node is proved if its value has been established to be true, whereas the node is disproved if its value has been determined to be false.A node is proved if its value has been established to be true, whereas the node is disproved if its value has been determined to be false. A node is solved as soon as it has been proved or disproved.A node is solved as soon as it has been proved or disproved. A tree is solved (proved or disprove) if its root is solved.A tree is solved (proved or disprove) if its root is solved. The goal of PN search is to solve a tree.The goal of PN search is to solve a tree.

10. PN-search (continue) Has high ability for solving AND/OR trees.Has high ability for solving AND/OR trees. However, it expands the whole search tree in working memory and require much memory.However, it expands the whole search tree in working memory and require much memory. This means that there is a severe memory restriction for hard problems.This means that there is a severe memory restriction for hard problems. The detail of PN search [Allis et al., 1994]The detail of PN search [Allis et al., 1994] PN^2 search (Breuker et al., 1999)PN^2 search (Breuker et al., 1999) Would it be possible to transform a best-first PN search into a depth-first search?Would it be possible to transform a best-first PN search into a depth-first search?

11. PN* (Seo, 1995) is recursive iterative-deepening depth-first search algorithm using proof numbers as a criterion to develop the frontier nodes.is recursive iterative-deepening depth-first search algorithm using proof numbers as a criterion to develop the frontier nodes. PN* starts searching by setting the proof-number threshold to 1 and expands the root node. If the proof number of the node under consideration exceeds the threshold, PN* stops node expansion at this node. When all frontier nodes have proof numbers exceeding the proof-number threshold, the search tree is discarded, the threshold is incremented and a new search starts.PN* starts searching by setting the proof-number threshold to 1 and expands the root node. If the proof number of the node under consideration exceeds the threshold, PN* stops node expansion at this node. When all frontier nodes have proof numbers exceeding the proof-number threshold, the search tree is discarded, the threshold is incremented and a new search starts. This iterative searching is enhanced by storing the expanded nodes and their properties in the transposition table.This iterative searching is enhanced by storing the expanded nodes and their properties in the transposition table.

12. PN* (continue) As the threshold is getting larger, node expansion proceeds in such a way that nodes having relatively small proof numbers are expanded first, like in best- first search, without the disadvantage of the memory requirement.As the threshold is getting larger, node expansion proceeds in such a way that nodes having relatively small proof numbers are expanded first, like in best- first search, without the disadvantage of the memory requirement. PN* uses the method of recursive iterative deepening. At an AND node the threshold values assigned to the child OR nodes are iteratively increased from 1 to their current maximum.PN* uses the method of recursive iterative deepening. At an AND node the threshold values assigned to the child OR nodes are iteratively increased from 1 to their current maximum. Enhanced by dynamic evaluation, efficient successor ordering and pruning by dependency relations.Enhanced by dynamic evaluation, efficient successor ordering and pruning by dependency relations. The detail of PN* [Seo, Iida and Uiterwijk, 2001]The detail of PN* [Seo, Iida and Uiterwijk, 2001]

13. Recursive iterative deepening threshold=2 threshold=3 threshold=4 recursive iterative deepening threshold=2 threshold=3 threshold=4 depth=1 depth=2 or 3 depth=3 or 5 depth=4 or 7 depth=5 or 9 best-first search (e.g. pn-search) iterative-deepening depth-first search frontier nodes iterative deepening at root node

14. Equivalence (1): AO* and PN* (Nagai, 1999) PN* (depth-first search) algorithm behaves the same as AO* (best-first search) in the meaning that PN* always expands a most-proving node.PN* (depth-first search) algorithm behaves the same as AO* (best-first search) in the meaning that PN* always expands a most-proving node. Most proving node (in the context of AO*) is a leaf node selected by tracing from the root in the following way.Most proving node (in the context of AO*) is a leaf node selected by tracing from the root in the following way. For each OR node, trace the child with minimum proof number.For each OR node, trace the child with minimum proof number. For each AND node, trace any child with non-zero proof number.For each AND node, trace any child with non-zero proof number.

15. Tsume-shogi solver: S EO Tsume-shogi solver: S EO based on the PN* algorithmbased on the PN* algorithm excellent management of transposition tableexcellent management of transposition table uses Dominance Relation of positionsuses Dominance Relation of positions omits the useless moves of dropping piecesomits the useless moves of dropping pieces other domain-specific featuresother domain-specific features solved “Microcosmos”, the problem with the longest steps, i.e., 1525 plies (1997)solved “Microcosmos”, the problem with the longest steps, i.e., 1525 plies (1997)

16. PDS (Nagai,1998) PDS ( Proof-number and Disproof-number Search ) is a straight extension of PN*, using both proof numbers and disproof numbers. PDS uses two thresholds in searching, the proof-number threshold and the disproof-number threshold.PDS ( Proof-number and Disproof-number Search ) is a straight extension of PN*, using both proof numbers and disproof numbers. PDS uses two thresholds in searching, the proof-number threshold and the disproof-number threshold. If the proof number or the disproof number exceeds the threshold at a certain node, PDS stops node expansion at this node. When PDS fails the root node expansion, it increases the either threshold value and restarts the searching.If the proof number or the disproof number exceeds the threshold at a certain node, PDS stops node expansion at this node. When PDS fails the root node expansion, it increases the either threshold value and restarts the searching. Note that the basic concept of PDS differs from PN search, i.e., asymptotically equivalent to PN search.Note that the basic concept of PDS differs from PN search, i.e., asymptotically equivalent to PN search.

17. DF-PN (Nagai,1999) Depth-first version of PN-searchDepth-first version of PN-search DF-PN first sets, at the root node, the thresholds of both proof number and disproof number to a certain large value (∞). Then, the threshold values are distributed among the descendant nodes.DF-PN first sets, at the root node, the thresholds of both proof number and disproof number to a certain large value (∞). Then, the threshold values are distributed among the descendant nodes. At every node, recursive iterative deepening is performed (cf. only OR nodes in PN*)At every node, recursive iterative deepening is performed (cf. only OR nodes in PN*)

18. Equivalence (2): PN and DF-PN (Nagai, 1999) DF-PN (depth-first search) algorithm behaves the same as PN (best-first search) in the meaning that DF-PN always expands a most-proving node.DF-PN (depth-first search) algorithm behaves the same as PN (best-first search) in the meaning that DF-PN always expands a most-proving node. Most proving node (in the context of PN) is a leaf node selected by tracing from the root in the following way.Most proving node (in the context of PN) is a leaf node selected by tracing from the root in the following way. For each OR node, trace the child with minimum proof number.For each OR node, trace the child with minimum proof number. For each AND node, trace the child with minimum disproof number.For each AND node, trace the child with minimum disproof number.

19. Tsume-shogi solver: N AGAI based on the DF-PN algorithmbased on the DF-PN algorithm efficient use of transposition tableefficient use of transposition table –SmallTreeGC and SmallTreeReplacement (Nagai, 99) pruning by dependency relations (positions and proof/disproof numbers)pruning by dependency relations (positions and proof/disproof numbers) –More efficient than the solver S EO GHI problem (cf. Breuker et al., CG98)GHI problem (cf. Breuker et al., CG98) Proof pieces (Seo, 99) and disproof piecesProof pieces (Seo, 99) and disproof pieces detecting a DAG to avoid the double counting of proof numbersdetecting a DAG to avoid the double counting of proof numbers

20. Performance of PN*, PDS and PN search (Sakuta and Iida, ACG9, 1999) Experiments in two domains:Experiments in two domains: –tsume-shogi (derived from GM’s games), and –6x6 othello (positions on PV) Comparison shows thatComparison shows that –In both, PDS outperforms PN*. –In othello, PN search variants do not show remarkable advantage over depth-first search. –In shogi, PN search variants show remarkable advantage over depth-first search. Forcing move sequences and sudden termination.Forcing move sequences and sudden termination. Disproving ability is important as well as proving.Disproving ability is important as well as proving.

21. Disproving ability compared in tsume-shogi

22. Four algorithms (equivalent)

23. More detail comparison Iterative deepening PNPN*PDSDF-PN behaviourdepth-firstbest-firstdepth-firstdepth-firstdepth-first transposition table yes basically no indispensableindispensableindispensable conspiracy number no proof & disproofproof disproof disproof iterative deepening depthnoproofthreshold proof & disproof disproof recursive iterative deepening nono at OR nodes at AND/OR nodes iteration at the root node yesyesproofthreshold proof & disproofno

24. Advantages of depth-first-type algorithm If a best-first algorithm and a depth-first algorithm behave in the same way, it is certainly better to adopt the depth-first one.If a best-first algorithm and a depth-first algorithm behave in the same way, it is certainly better to adopt the depth-first one. –IDA* and A*, MT-SSS* and SSS*, and –MTD(f) for minimax trees [Plaat et al., 1996] Many enhancements can be used:Many enhancements can be used: –History heuristics, transposition table, iterative deepening, and so on. Depth-first algorithm uses less memory spaceDepth-first algorithm uses less memory space –Search speed is often faster in practice

25. Summary of experimental performance PN* solved most hard problems, among which one with 1525 steps. But, it was unable to solve 10 hard problems with long steps over 300.PN* solved most hard problems, among which one with 1525 steps. But, it was unable to solve 10 hard problems with long steps over 300. PDS basically outperforms PN* in solving long-step problems.PDS basically outperforms PN* in solving long-step problems. DF-PN have solved all hard-problems.DF-PN have solved all hard-problems.

26. Application to actual endgames in shogi Advances of AND/OR tree search algorithms enable a computer to:Advances of AND/OR tree search algorithms enable a computer to: –Find a mate under tournament time constraints, at interior nodes (close to the root). –Reuse the outcome of mating search in the successive searches. –Find a brink-mate (forcing mate threat) –Play a high-level endgame (mating race).

27. Conclusions Depth-first searches using proof/disproof numbers have recently been developed.Depth-first searches using proof/disproof numbers have recently been developed. –Basically depth-first search but behaves in best-first manner like proof number search (PN search). –Memory requirement is much less than best-first. DF-PN have solved all hard-problems of tsume- shogi with long steps over 300.DF-PN have solved all hard-problems of tsume- shogi with long steps over 300. These algorithms are now used in shogi playing programs.These algorithms are now used in shogi playing programs. –High-level mating race in the endgame.