1. D1 Miwa Makoto Chikayama & Taura Lab
Automatic Construction of Static Evaluation Functions for Computer Game Players
D1 Miwa Makoto
Chikayama & Taura Lab

2. Outline Introduction Static Evaluation Functions
Computer Game Player
Static Evaluation Functions
Automatic Construction of Static Evaluation Functions
GLEM
Static Evaluation Functions from domain theory
Conclusion & Future Work

3. Computer Game Player Game Computer Game Player
2-player, zero-sum, deterministic, and perfect information game
ex. chess, Shogi, Go, etc…
Computer Game Player
receives a position and returns the next move
Important Features
Evaluation Functions
Game Tree Search (& search strategy)

4. Static Evaluation Functions (SEFs)
Static ≒ without search
Evaluate positions and assign their heuristic values
Probability to win
Distance to goal …
E(p) = number of lines open for Ｘ –
number of lines open for O
E(p) = 3 – 2 = 1 Evaluation Value

5. Static Evaluation Functions (SEFs)
Evaluation value
Rank treating positions
WIN +1
Static
Evaluation
Function +2 +1 -1 -2 -1
LOSE

6. Game Tree Search Static evaluation function
MAX
Tic Tac Toe with
horizon = 2 1
Static
evaluation
function -1
MIN 1
Static evaluation fuctions decide
the computer game players’ behavior 1 -1 2 -2

7. Construction of SEFs Two requirements Representation of SEFs
evaluation accuracy
speed
Representation of SEFs
Combinations of features
feature
Number of pieces, Position, etc…
trade-off

8. Construction of SEFs To write SEFs by hand, developers need ...
Deep knowledge about the treating game
To find useful features and essential features
Much effort to tune
To avoid oversights
ex. self-play, play with other players
What if no experts exist?
What if the player is stronger than human experts?
Automatic construction of SEFs

9. Automatic Construction of SEFs
Construction from simple features
Simple features
A set of units to represent a position
ex. Black stone on A1 (Othello)
2 kinds of methods
Direct Method
Hybrid Method

10. Automatic Construction of SEFs
Direct Method
High-level combination of simple features
Genetic Programming, Neural networks, etc…
Feature
High expressivity
Much resources
Difficult to analyze
Ex.
TD-Gammon, The Distributed Chess Project

11. Automatic Construction of SEFs
Hybrid Method
Linear combination of high-level features from simple features
Feature (in comparison with direct method)
Less expressive
Less resources
Easy to analyze and optimize constucted SEFs
Fast SEFs
Ex.
ZENITH, GLEM

12. GLEM (M. Buro, 1998) Generalized Linear Evaluation Model
Application : Othello
Logistello
One of the strongest computer Othello players
won the human champion in 1997

13. Method Extraction of atomic features
Generation of configurations by conjunction of features
Weighting configurations using linear regression

14. Atomic features Simple binary features Extracted by hand
ex. Black stone on A1
Extracted by hand
ex. Othello
2£64 features - places and their stones
Can not be expressed by a combination of other features.

15. Configurations
Extracting all conjunctions of atomic features is unreasonable
ex. Othello conjunctions
Extract only frequent conjunction
(=Configurations)
frequent
at least N times in the training set

16. Patterns in LOGISTELLO
Treating the whole position at once costs high
Extract a part of the position by hand
Pattern values are pre-computed and preserved in a hash table
 Fast Evaluation
Patterns in LOGISTELLO

17. Patterns in LOGISTELLO
Evaluation using hash tables
Patterns in LOGISTELLO

18. Application : Othello 13 game stages (every 4 discs)
Sum of 51 pattern values
1.4 million positions/sec on Athlon 2000+
1.5 million weights
17 million training positions
10x speedup
compared with the old one which won the human champion in 1997

19. SEFs from domain theory (Kaneko et al. 2003)
Based on ZENITH (Fawcett 1993)
Application : Othello

20. SEFs from domain theory
Generation of logical features from domain theory
Extraction of evaluation features from logical features and selection
Weighting evaluation features using linear regression

21. Domain Theory
Rules and goal conditions written by a set of Horn Clauses
ex. Othello

22. Logical Features Features generated from domain theory
Generation Strategy
Translate domain theory
Remove expensive features
Select features by backward elimination method

23. Logical Features Translations Decomposition split conjunction
remove negation
split arithmetic comparison
split arithmetic calculation
Abstraction
remove least constraining term
remove a variable
Goal Regression
regress the formula
Specialization
remove a feature’s disjunction
replace call to recursive predicate with base case
find invariant variable values

24. Evaluation Features Evaluation Features Boolean value
Conjunction of facts
ex. blank(a1) Æ owns(x, a2) Æ owns(o, a3)
(=white can play on square a1)
fact
a clause without body
ex. owns, blank

25. Extraction of Evaluation Features
Translate logical feature to patterns.
logical feature
f(A) :- legal_move(A, o).
unfolding
unfolded features
legal_move(a1, o) :- blank(a1), owns(x, a2), owns(o, a3).
legal_move(a1, o) :- blank(a1), owns(x, b1), owns(o, c1). ….
extract conjunction of body
Evaluation Features
blank(a1) Æ owns(x, a2) Æ owns(o, a3)
blank(a1) Æ owns(x, b1) Æ owns(o, c1) ….

26. Selection of Evaluation Features
Frequency
Reject high- and extreme low-frequency evaluation features
Approximated Forward Selection
Select high correlation pattern iteratively

27. Fast access to features
Hasse diagram
Kernel Extraction

28. Experimental Results 11,079 logical features
8,592,664 evaluation features without selection
800,000 positions for training and 6,000 positions for testing
Positions of 55 and 60 discs
Range of evaluation values : [-64, 64]

29. number of evaluation features in evaluation functions
Accuracy
number of evaluation features in evaluation functions
as well or better than GLEM in accuracy

30. number of evaluation features in evaluation functions
Speed
Athlon MP 2100+
30,000 positions/sec (1.4 million in GLEM)
number of evaluation features in evaluation functions

31. Conclusion Automatic Construction of Static Evaluation Functions GLEM
SEFs from domain theory

32. Future Work More sophisticated feature selection
Automatic extraction of GLEM’s pattern
More expressive combination of simple features