1. Knowledge acquisition for adative game AI Marc Ponsen et al. Science of Computer programming vol. 67, pp. 59-75, 2007 장수형

2. Outline Introduction Related work Adaptive Script of Wargus Experiment Result Alternative method 2 /24

3. Introduction Game –Become increasingly realistic –Graphical presentation –Capabilities of characters ‘living’ Game AI –Game developers Encompass techniques such as pathfinding, animation, collision physics –Academic researchers Intelligent behavior –Inferior quality Benefit from academic research into commercial games 3 /24

4. Introduction Adaptive game AI –Behavior of computer-controlled opponents –Potentially increase the quality of game AI –Incorporate a sufficient amount of correct prior domain knowledge Dynamic scripting –Offline reinforcement learning technique –Dynamic scripting in a real-time strategy game called Wargus –Ambitious performance task –The quality of the knowledge base is essential 4 /24

5. Introduction Knowledge base –Manually encode Take a long time Sub-optimal due to analysis Not generate satisfying result –Semi-automatically Increase the performance Machine learning Added to knowledge bases Evolution algorithm –Automatically Evolutionary algorithm Automatically transfers the domain knowledge 5 /24

6. Related work Few studies exist on learning to win complex strategy games Focusing on simpler tasks –Relational Markov decision process model to some limited Wargus scenarios(Guestrin et al.) –Case-bases plan recognition approach for assisting Wargus player(Cheng and Thawonmas) Manual knowledge acquisition –Typical RTS games(Age of Empires and Command & Cunquer) Semi-automatic knowledge acquisition –Pattern recognition technique(Street et al.) Automatic knowledge acquisition –Neural network for Backgammon, GO, Chess(Kirby) 6 /24

7. RTS games Usually focus on military combat Control armies and defeat all opposing forces that are situated in a virtual battlefiled(often called a map) in real-time Collecting and managing resources Determines all decision for a computer opponent over the course of the whole game –Form of scripts which are list of game action that are executed sequentially –Constricting buildings, researching new technologies, and combat 7 /24

8. Wargus Clone of the popular RTS game Warcraft II Open source Stratagus engine Strategy –Small Balanced Land Attack –Large Balanced Land Attack –Soldier’s Rush –Knight’s Rush 8 /24

9. Complexity of Wargus No single tactic dominates all others –The rock-paper-scissors principle Large action space –The set of possible actions that can be executed at a particular moment In Wargus… –A : number of assignments workers can perform –P : average number of workplace –T : number of troops –D : Average number of directions that a unit can moves –S : number of choices for a troop’s stance –B : number of buildings –R : average number of choices for research objectives at a building –C : average number of choice of units to create at a building 9 /24

10. Complexity of Wargus 10 /24

11. Complexity of Wargus Decision complex of each state – –Higher than the average number of possible moves in many board game such as chess(30) 11 /24

12. Dynamic Scripting for Wargus Game AI for complex games is mostly defines in scripts –Contain weaknesses, which human players can exploit –Dynamic script Introduced by Spronck et al. Ability to adapt to a human player’s behavior The probability that a tactics is selected for a script is an increasing function of its associated weight value –Requirements The game AI can be scripted Domain knowledge on the characteristics of a successful script can be collected Evaluation function can be designed to assess the success of the function’s execution 12 /24

13. Dynamic Scripting for Wargus Divide the game into a small number of distinct game states Each state corresponds to a unique knowledge base 13 /24

14. Weight adaptation in Wargus F : The overall fitness F i : the stats fitness(state i) S d : the score for the dynamic player S o : the score for the player’s opponent 14 /24

15. Weight adaptation in Wargus S x : the score of the dynamic player state x M x : the military points for player x B x : building points for player x 15 /24

16. EA(Fitness Function) M d : Military points for the dynamic player M o : Military points for the dynamic player’s opponent b : break-even point C t : game cycle C max : maximum game cycle(the longest time a game is allowed to continue) 16 /24

17. EA(Encoding) Construct, research, economy, combat genes.. 17 /24

18. Performance evaluation Dynamic scripting under three condition –Manually acquired –Semi-automatically acquired –Automatically acquired The other is controlled by a static script Four strategy –SBLA, LBLA, SR, KR Randomization turning point –Number of the first game in which the dynamic player statistically outperforms the static player –A low RTP value indicates good efficiency 18 /24

19. Result 19 /24

20. Conclusions Three alternative for acquiring high-quality domain knowledge used by adaptive game AI –Manual, semi-automatic, automatic Discussed dynamic scripting Domain knowledge is crucial factor to the performance of dynamic scripting The automatic knowledge acquisition approach takes best performance 20 /24

21. Alternative method Alternative method of script handling –Bayesian Network Case study : StarCraft ‘Adaptive Reasoning Mechanism with Uncertain Knowledge for Impro ving Performance of Artificial Intelligence in StarCraft 21 /24

22. 상성파악 전략과 유닛의 상성 파악 22 /24

23. 베이지안 네트워크 설계 불확실한 지식정보 – 상대방 진영으로의 정찰 시도 – 지어진 건물들의 구성 – 생산한 유닛의 구성 – 건물과 유닛의 개수 – 위의 정보들을 얻어낸 시각 거짓정보는 아니지만 완벽한 정보도 아니다 – 숨겨진 유닛, 숨겨진 건물, 지어지다가 취소된 건물 23 /24

24. 스크립트 선택 정보 추론 후 가장 효과적인 대응 스크립트 선택 24 /24

25. 결과 실험결과 25 /26

26. E.N.D