1. Kriegspiel Jason Wolfe

2. Shall I fire on them now, sir? Not yet Kif; in the game of chess, you can never let your adversary see your pieces

3. Outline Introduction Rules of Kriegspiel Rules of Kriegspiel Our Problem Our Problem Sample Kriegspiel Mating Problem Sample Kriegspiel Mating Problem Data Structures and Algorithms Exact Belief State Exact Belief State Exact Mating Algorithm Exact Mating Algorithm Efficient Belief State Implementations Efficient Belief State Implementations

4. Rules of Kriegspiel Same initial board, movement and mating rules as ordinary chess Difference: we can’t see our opponent’s pieces, and we only know the consequences (capture, check) of our opponent’s moves Playing Kriegspiel requires three chess boards and a referee (or online play) Playing Kriegspiel requires three chess boards and a referee (or online play)

5. More Rules Since players have limited knowledge of their opponents’ pieces, they don’t know in advance which chess moves will be legal Possibly-legal moves can be illegal iff they are obstructed by one of the opponent’s pieces, or leave the player in check Possibly-legal moves can be illegal iff they are obstructed by one of the opponent’s pieces, or leave the player in check On his turn, a player keeps attempting possibly-legal moves until one is accepted as legal; this becomes his actual move for the turn On his turn, a player keeps attempting possibly-legal moves until one is accepted as legal; this becomes his actual move for the turn Thus, a “turn plan” in Kriegspiel is a (truncated) permutation of the possibly-legal moves Thus, a “turn plan” in Kriegspiel is a (truncated) permutation of the possibly-legal moves Stalemate rules (position 3-peat and 50-moves) suspended because position includes knowledge about opponent’s pieces

6. More Rules The referee is the only source of information about the opponent’s pieces. For each attempted move, he announces “Yes”, “No”, or “Impossible” For captures, he announces the square at which capture occurred (but does not disclose the identity of either piece) For checks, he announces check and the direction of check (row, column, long diagonal, short diagonal, or knight) Checkmates and stalemates are announced, and the full board position is revealed to both players

7. Basic Strategy While the general objective is the same as chess (capture the opponent’s pieces and corner his king), uncertainty introduces several subsidiary objectives Gathering information about the opponent’s pieces Gathering information about the opponent’s pieces Hiding one’s own pieces from the opponent Hiding one’s own pieces from the opponent

8. More Basic Strategy The effects of these factors can be seen even in the first few moves of a real Kriegspiel game Less typical openings may be desirable, because they may not be anticipated Less typical openings may be desirable, because they may not be anticipated Power pieces should always be protected if possible Power pieces should always be protected if possible In general, all possible pawn captures should be attempted before trying other moves In general, all possible pawn captures should be attempted before trying other moves Thus, we should be careful playing power pieces around probable locations of opponent’s pawns Many variants include an ‘Any’ rule to speed up gameplay Special care needs to be taken to protect against short mates such as Fool’s mate. Special care needs to be taken to protect against short mates such as Fool’s mate.

9. Specifics of Our Problem Specifically, we will be given a move history for which White has 5 pieces on the board White has 5 pieces on the board Black has 2 pieces on the board Black has 2 pieces on the board It is White’s turn to move, and given the move history a certain checkmate exists within 2.5 moves It is White’s turn to move, and given the move history a certain checkmate exists within 2.5 moves Our problem is to discover a conditional move plan that is certain checkmate This talk is focused on the goal of locating certain checkmate. This approach differs significantly from that for locating probable checkmates. This talk is focused on the goal of locating certain checkmate. This approach differs significantly from that for locating probable checkmates.

10. More Specifics Our problem is not as complicated as ordinary Kriegspiel We are concerned with certain checkmates, which guarantee checkmate regardless of how our opponent plays We are concerned with certain checkmates, which guarantee checkmate regardless of how our opponent plays Because we don’t need to take probability into account, we don’t need to consider our opponent’s strategy Because we don’t need to take probability into account, we don’t need to consider our opponent’s strategy This means that (at least for fixed-depth mates) we can assume that the opponent can see our pieces (but not visa-versa) This means that (at least for fixed-depth mates) we can assume that the opponent can see our pieces (but not visa-versa)

11. Two Problems 1.Discover (approximate) the current belief state given the move history 2.Search the game tree for certain checkmate, given the belief state calculated in (1)

12. Example Kriegspiel Checkmate White to move and mate within 1.5 moves. Given the move history to this point, Black’s pieces are known to be in one of the above three possible positions.

13. White’s Move Checkmate No Try Qb8

14. YesKnight Check White’s Move Try Ne7 Checkmate

15. Black’s Move Yes Opponent’s move(s)

16. White’s Move Checkmate Mate with Qh2 Mate with Qf8

17. Opponent’s move(s) Initial Belief State Check- mate No Try Qb8 Knight Check Try NeE7 Yes Mate with Qh2Mate with Qf8 Check- mate Checkmate

18. Basic Kriegspiel Mates In ordinary chess, a king and rook or greater can always checkmate a lone king. Interestingly, the same elementary mates are possible in Kriegspiel Two elementary mates require randomization King, bishop, and knight vs. king has a certain mate, but requires a possibly infinite number of moves King, bishop, and knight vs. king has a certain mate, but requires a possibly infinite number of moves King and two bishops vs king has an epsilon-mate King and two bishops vs king has an epsilon-mate

20. Kriegspiel Data Structures and Algorithms

21. Belief States The primary difference between Kriegspiel and chess is that in Kriegspiel, players must maintain a belief state of possible positions the opponent could be in The simplest belief state representation is a list of possible board positions In a real Kriegspiel game, a belief state is a possibility distribution rather than a list In a real Kriegspiel game, a belief state is a possibility distribution rather than a list

22. Exact Belief State Operations on an exact belief state are simple The initial state is simply the initial board position The initial state is simply the initial board position When a player makes a move, he prunes positions from the belief state that are inconsistent with the referee’s response When a player makes a move, he prunes positions from the belief state that are inconsistent with the referee’s response For the opponent’s move, the belief state expands to the union of the successors of its component states, and is then pruned based on the referee’s response For the opponent’s move, the belief state expands to the union of the successors of its component states, and is then pruned based on the referee’s response When an exact belief state is split based on possible referee responses, the sum of the cardinality of the component belief states equals the cardinality of the original state (perfect information) However, this representation is intractable since a real belief state could easily contain more than 10 10 possible positions

23. Exact Mating Algorithm The exact belief state precisely captures what positions the opponent could be in, given some move history The other part of the problem is mating search, given such a belief state As we will see, this problem also has a relatively simple exact solution As we will see, this problem also has a relatively simple exact solution

24. Exact Mating Algorithm (cont.) The Kriegspiel exact mating algorithm 1.Branch (OR) over possible turn plans, looking for a certain checkmate 2.Branch (AND) over possible referee responses, fragmenting the belief state 3.If a sub-belief state contains stalemate, it is a loss. If it contains only checkmated positions, it is a win. Otherwise, if we are not yet at the maximum search depth, prune terminal positions from the belief state and continue search 4.Set the belief state to the union of its successor positions (simulating the opponent’s move) 5.Branch (AND) over possible referee responses, fragmenting the now-expanded belief state 6.If a sub-belief state contains checkmate or stalemate, it is a loss. Otherwise, go to step 1

25. Game Tree Opponent’s move(s) Initial Belief State Check- mate No Try Qb8 Knight Check Try Ne7 Yes Mate with Qh2Mate with Qf8 Check- mate Checkmate

26. Exact Mating Algorithm (cont.) This search has three major branch points Choosing a “turn plan”, a truncated permutation of the possibly-legal moves Choosing a “turn plan”, a truncated permutation of the possibly-legal moves The referee’s response to the legal move The referee’s response to the legal move The referee’s response to the opponent’s move The referee’s response to the opponent’s move Since we may choose our turn plan but not the referee’s responses, the first branch is an OR- search and the other two are AND-searches. We do not have to explicitly branch based on the opponent’s move, since this branching occurs within the belief state

27. An Initial Problem With 5 pieces at an average of 10 possible moves each, the first branching factor could be on the order of 10 30 possible turn plans to check If we know a move is illegal, it prunes our belief state; therefore, it seems that we must check all permutations If we know a move is illegal, it prunes our belief state; therefore, it seems that we must check all permutations

28. The Solution We are not just looking for the turn plan with maximum utility any turn plan with utility less than 1 is unacceptable any turn plan with utility less than 1 is unacceptable If a “turn plan” contains a move which, if accepted, is not certain checkmate, it will not have utility 1

29. The Solution (cont.) This suggests the following simple algorithm for choosing a turn plan, which has a branching factor of only m 2 rather than m! : Start with an empty move plan and some belief state BS Start with an empty move plan and some belief state BS Loop until BS does not change (failure) or is empty (success) Loop until BS does not change (failure) or is empty (success) Loop through possible moves If the move is certain checkmate if accepted, add it to the move plan and prune its acceptance set from BS If the move is certain checkmate if accepted, add it to the move plan and prune its acceptance set from BS

30. Branching Factors 1.Optimistically, with 50 possible moves this branching factor might be as small as 100 2.There are 2-10 possible referee responses for our move 3.There are 1-200 possible responses for our opponent’s move

31. Heuristic improvements The first stage can be improved by heuristic move ordering Sizes of move’s acceptance sets Sizes of move’s acceptance sets Heuristic likelihood move will cause mate Heuristic likelihood move will cause mate The second and third (AND) searches can be improved with heuristic state ordering Larger belief states are less likely to be certain checkmate Larger belief states are less likely to be certain checkmate

32. Current Status My current Kriegspiel code, which utilizes the tools described so far, can solve the “example mate” in under a minute

33. Another Improvement If a belief state admits a state that is not checkmate in ordinary chess within the necessary depth, it does not admit certain Kriegspiel checkmate Thus, a good tactic might be to sample a few positions (heuristically) before running the tree search on the full belief state mating search in ordinary chess has significantly lower branching factors and amount of work per operation mating search in ordinary chess has significantly lower branching factors and amount of work per operation

34. Efficient Belief States With the exception of further heuristic improvement and sampling techniques, the mating algorithm is essentially finished The simple belief state representation, on the other hand, is not tractable for real Kriegspiel problems Thus, what we need to do now is develop a compact, accurate, and fast belief state implementation

35. Belief State Methods For mating search, the belief state needs to support the following four basic methods: Make a specific move for player, and fragment the belief state based on the referee’s possible responses Make a specific move for player, and fragment the belief state based on the referee’s possible responses Make all possible moves for opponent, and fragment the belief state based on the referee’s possible responses Make all possible moves for opponent, and fragment the belief state based on the referee’s possible responses Test whether a belief state with player-to-move contains any checkmate or stalemate positions Test whether a belief state with player-to-move contains any checkmate or stalemate positions Test whether a belief state with opponent-to-move contains any stalemate positions or all checkmate positions; if neither, prune checkmate positions Test whether a belief state with opponent-to-move contains any stalemate positions or all checkmate positions; if neither, prune checkmate positions

36. Unary Marginals For move generation and (especially) terminal testing, a positional representation of the belief state is necessary However, the exact positional representation is intractably large A first compromise is to store only the “unary marginals”, the sets of possible board positions (including off-the-board) that each of the opponent’s pieces could possibly occupy This approximation allows for a sound, but not complete, algorithm for discovering certain checkmate This approximation allows for a sound, but not complete, algorithm for discovering certain checkmate

37. Exact Belief State

38. Unary Marginal Belief State

39. Example (cont.) The unary marginal approximation to the exact belief state admits 11 positions Of these, only the original 3 positions are ordinary-chess checkmates! Of these, only the original 3 positions are ordinary-chess checkmates! Even if all the “decoy” states were ordinary-chess checkmates, there might be no Kriegspiel checkmate given this belief state

40. Unary Marginal Characteristics Storing the unary marginal belief state requires only 16 * 65 = 1040 bits Move generation for us (White) is simple, because we know the positions of our own pieces exactly Move generation for Black can also be done in constant time But, the belief state will mix quickly. Pieces will only block others if their positions are known certainly But, the belief state will mix quickly. Pieces will only block others if their positions are known certainly In general, when we split the belief state based on the referee’s response, the sum of the sizes of the component belief states will be larger than the size of the original belief sate (imperfect information)

41. Information in Kriegspiel In Kriegspiel, the only way we can learn about the opponent's pieces is through the referee’s responses (bolstered by our knowledge of the rules of the game and the locations of our own pieces) Thus, before proceeding, we should analyze exactly what kinds of information we get from the referee, and see how the unary marginal belief state can handle these kinds of information

42. Information in Kriegspiel What kinds of information can we get from the referee? Our opponent... has/does not have some specific piece at some specific location has/does not have some specific piece at some specific location has/had a (unspecified) piece at a specific location has/had a (unspecified) piece at a specific location has at least one (unspecified) piece at some range of locations has at least one (unspecified) piece at some range of locations has a piece attacking a specific space (sometimes from a specific direction) has a piece attacking a specific space (sometimes from a specific direction) has no unblocked piece attacking a specific space has no unblocked piece attacking a specific space

43. Information in Kriegspiel Only the first of the five types of information can be captured by unary marginals Thus, if possible we would like to amend this representation to capture a higher proportion of the information

44. Amending the Belief State The unary marginals store 16 bits for each square (incl. off-the-board) Each bit stores whether a specific piece of Black’s could possibly be at this position Each bit stores whether a specific piece of Black’s could possibly be at this position As a very simple refinement, simply store more bits per square to try to capture more information A single bit can store whether or not a square is certainly occupied A single bit can store whether or not a square is certainly occupied Two more bits let us store whether a square is certainly/possibly/impossibly attacked by an opponent’s piece Two more bits let us store whether a square is certainly/possibly/impossibly attacked by an opponent’s piece Six more bits let us store whether a square is certainly occupied by an opponent’s piece that attacks in a certain direction (row/column or diagonal), or by a specific type of piece (knight, pawn, bishop, or rook) Six more bits let us store whether a square is certainly occupied by an opponent’s piece that attacks in a certain direction (row/column or diagonal), or by a specific type of piece (knight, pawn, bishop, or rook)

45. Amended Belief State With the additions from the previous slide, the belief state can capture all but has at least one (unspecified) piece at some range of locations has at least one (unspecified) piece at some range of locations While the extra information will be useful in immediate reasoning, it is fairly transient

46. Another Refinement The unary marginals store, for each piece, what squares it could possibly occupy This concept can be extended to the “binary marginals” by storing, for each pair of pieces, what pairs of squares they could possibly occupy Among other things, this allows us to capture when Black’s pieces block one-another However, this representation requires (65*16) 2 bits to store, and still ignores many basic constraints This can be extended up to 16-ary marginals, which are identical to the exact belief state.

47. Some Problems Although they seem like a reasonable approach, the “n-ary marginal belief states” are far from perfect: They admit … Illegal board positions Illegal board positions Inconsistent positions (multiple pieces per square) Inconsistent positions (multiple pieces per square) Positions where the opponent has the incorrect number of pieces Positions where the opponent has the incorrect number of pieces Positions where the opponent has made no moves at all, or all possible moves simultaneously Positions where the opponent has made no moves at all, or all possible moves simultaneously

48. Implementation of Unary Marginals Currently, I am in the process of implementing the unary marginal belief state Move generation and pruning based on the referee’s response can be accomplished fairly simply (with limited accuracy) Move generation and pruning based on the referee’s response can be accomplished fairly simply (with limited accuracy) We only need to look at the possible locations of one or two pieces at a time Unfortunately, this does not seem to be true of terminal test Unfortunately, this does not seem to be true of terminal test

49. Terminal Test Issues We are searching for certain checkmate Thus, if the belief state at any point contains a stalemate position, we fail However, detecting stalemate from unary marginals is difficult Stalemate typically depends on the locations of all the pieces on the board Stalemate typically depends on the locations of all the pieces on the board No algorithm (that I can think of) can exactly detect stalemate positions in a marginal belief state, without fully enumerating at least some of its positions No algorithm (that I can think of) can exactly detect stalemate positions in a marginal belief state, without fully enumerating at least some of its positions

50. Proposed Implementation The best implementation I can come up with for terminal test is a depth first search on possible positions admitted by the belief state With some intelligent pruning and ordering techniques, this will hopefully be tractable for belief states of moderate size.

51. Illustrating the Approach Implementing the terminal test method requires four sub-algorithms I will illustrate the approach using the problem of discovering stalemate with Black to move This approach has two steps: Fix the board so that Black cannot be in check Fix the board so that Black cannot be in check Search for mate (a position with no legal moves) within this board position Search for mate (a position with no legal moves) within this board position

52. Step 1 For each square the Black king could be placed not in check, AND For each pair of squares that the Black king could be placed in check and another Black piece could be placed to block this check, Fix these pieces at the chosen locations Fix these pieces at the chosen locations Do a depth first search on this modified belief state for positions with no legal moves. If such a position is found, return true. Do a depth first search on this modified belief state for positions with no legal moves. If such a position is found, return true.

53. Step 2 In the depth first search for positions with no legal moves, we fix the position of one piece at each branch For a position to be stalemate, all of Black’s possible moves must either be Blocked by other Black pieces Blocked by one of White’s pieces (Black’s pawns only) King moves that place Black’s King in check Other moves that uncover check against Black’s King

54. Pruning Consider some partial placement of Black's pieces that contains legal moves for Black. Since White's positions are already known, the only way we can produce a stalemate is by placing more of Black's pieces where they block the currently possible moves by Black (and fail to create more possible moves). Since White's positions are already known, the only way we can produce a stalemate is by placing more of Black's pieces where they block the currently possible moves by Black (and fail to create more possible moves). Thus, for any partial placement of Black's pieces, we can count the minimum number of other Black pieces that must be placed to make this a stalemate; call this number P (as a first approximation, we can pretend that placing a piece will not create any new legal moves). Thus, for any partial placement of Black's pieces, we can count the minimum number of other Black pieces that must be placed to make this a stalemate; call this number P (as a first approximation, we can pretend that placing a piece will not create any new legal moves).

55. Pruning Then, the pruning itself is actually very simple: If Black has fewer pieces left to be placed on the board than P, prune away this branch. One can imagine that this will be very common, especially if high-mobility pieces like the knight are placed first. This pruning also suggests a simple position ordering; try positions that have the lowest P first, because these are closest to mate.

56. Evaluation The other three terminal test sub-methods differ significantly, but follow the same general approach This approach allows for re-incorporating the lost constraints discussed earlier, efficiently pruning: Inconsistent positions (multiple pieces per square) Inconsistent positions (multiple pieces per square) Positions where the opponent has the incorrect number of pieces Positions where the opponent has the incorrect number of pieces Positions where the opponent has made the incorrect number of moves Positions where the opponent has made the incorrect number of moves However, it remains to be seen whether it is accurate and efficient enough to be useful

57. Looking Forward... We still have a few significant hurdles before we get to the point of solving the challenge problems: Accurately approximating a belief state, given a move history Accurately approximating a belief state, given a move history Locally refining “important” areas of our belief state? Locally refining “important” areas of our belief state? Requires detecting “near-mates” Improving the belief state implementation Improving the belief state implementation

58. Questions?