1. Machine Learning for Go
Jung-Yun Lo Dept. of computer science and information engineering National Dong Hwa University

2. Outline
A survey of the application of machine learning to the game of Go
A learning architecture for the game of Go

3. A survey Some possible directions of research
Global approaches
Learning in search
Learning in the endgame
Learning in the opening
The representation language

4. Global approaches
Learning a function from a board position and a move to a reward
On large boards, probably a more specific method should be used for different subproblems of the game

5. Learning in search temperature (high → deeper search)
candidate move ordering
temperature (low → stop search)
Leaf node static evaluation

6. Learning in the endgame
Each local endgame positions is evaluated, then the whole game is considered as a sum of games
Decomposition search

7. Learning in the opening
Hard to quantify
Using joseki
Depend on the surrounding situation
Learning a global rules for opening moves

8. The representation language
Difficult to express more high-level concepts such as liberty, atari, ladder and eye
Making the representation language more expressive

9. The representation language
block( BlockID, Color, Size, LibertyCount)
board( X, Y, GroupID)
adjacent( BlockID1, BlockID2)

10. The representation language

11. The representation language
The Common Fate Graph (Enzenberger, 1996)

12. The representation language
projection
projection

13. The representation language
Loss of information in the CFG

14. Conclusions
Learning result are promising, but the whole field is nearly unexplored and much opportunities to do research

15. A learning architecture for the game of Go
Combinatorial Game Theory
The HUGO Architecture
Three Components of HUGO
Choice of Subgames
Initiative Engine
Computing Game Value

16. Combinatorial game theory
G = {F|O}
F : the set of options that player Friend can reach with one legal move
O : for player Opponent
F can be 2 possible value
W : win for Friend
L : loss for Friend

17. Combinatorial game theory
4 possible outcomes for a combinatorial game : WW, WL, LL, and LW
WW : won by Friend, irrespective of who moves first
WL : an unsettled game, won by the player who moves first
LL : lost for Friend even if Friend moves first
LW : the player who moves first will loss the game

18. The HUGO architecture
Can be applied to any 2-player, deterministic, full information, partizan, combination game

19. 3 components of HUGO Choice of subgames Initiative engine
Select a collection of well-defined subgames
Ensure a high discriminative abilities
Initiative engine
Find the move that yields the most points
Prefer holding initiative
Computing game values
Compute the game-theoretic value of a particular game

20. Future work
Study more reference about machine learning

21. Reference
A Survey of The Application of Machine Learning to The Game of Go Jan Ramon, Hendrik Blockeel / Katholieke Univ. Leuven
A Learning Architecture for The Game of Go A.B. Meijer, H. Koppelaar / Delft Univ. of tech.
Computer Go and Machine Learning Thore Graepal