CS510 AI and Games Final Report on Dec Juncao Li

Science, Portland State University2 Agenda About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion

Science, Portland State University3 Advanced Protection (AP) Author: Soren Johnson Turn-based strategy game –Player makes strategy each turn –AI plays against player’s strategy Static AI V.S. adaptive AI –Player learns the strategy of static AIs –Adaptive AI adapts based on users performance

Science, Portland State University4 AP (Cont.)

Science, Portland State University5 AP (Cont.)

Science, Portland State University6 AP (Cont.)

Science, Portland State University7 AP (Cont.) Features –Each minion has a brain (automaton) encoded by a 128-bit string –Four behaviors depending on the input: Move forward, turn right, turn left and do actions –250 brains for a minion to choose 20 hardcoded, 230 from genetic algorithm During the play, brains are rated based on their performance against the player –Dynamically choose best-fit brains Each player has their own easy and hard minion brains Players does not loose or win too much

Science, Portland State University8 Minion Inputs

Science, Portland State University9 Why Brain Matters

Science, Portland State University10 Why Brain Matters (Cont.)

Science, Portland State University11 Why Brain Matters (Cont.)

Science, Portland State University12 Agenda About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion

Science, Portland State University13 My Goal & Why Design AIs that play as players –Try to learn the player’s strategy –Try to perform best on all 250 brains Because Chaos selects brains based on player’s strategy Win (defined later) brains as much as it could Design benchmark for the AIs on each side –How well each AI performs Learning about players helps game designs –Player will not like a game that they can never win! –Player will not like a game that they can easily win!

Science, Portland State University14 Finite State Machine (FSM) Code the game strategy in a FSM Inputs or what matters –Treasury: Human and Chaos –Terrain and current human nodes on the map Output –Where to place the units –What units to place –Based on current state Allocate more farmers for earning money Allocate more infantry for attacking Chaos

Science, Portland State University15 FSM (Cont.)

Science, Portland State University16 FSM (Cont.)

Science, Portland State University17 FSM (Cont.)

Science, Portland State University18 FSM (Cont.) My FSM is static Hard to make it dynamic and adaptive –Too many possibilities (24*24 size map plus human ……) Works well as a benchmark –How well the Chaos AI performs Train Neural Network (NN) –Hopefully, teach the NN to recognize the map

Science, Portland State University19 Use Neural Network (NN) Input nodes: MAP_SIZE+2 –Map size: 24*24 = 576 –Human+Chaos treasury: 1+1 Output nodes: MAP_SIZE –Human units placement One hidden layer: MAP_SIZE+2 Normalize the inputs/outputs range: [0.0, 1.0] Use other’s NN code: –AI Game Engine: well developed, but not efficient –Tim Jones book: simple and efficient NN

Science, Portland State University20 Train the NN Firstly, use the static FSM to train the NN –Random generated maps –Random initial Human/Chaos treasury –FSM generates outputs as the training case Secondly, improve the NN –Randomly generate strategy –If it performs better than current NN –Train NN with the strategy

Science, Portland State University21 Understand the NN Outputs NN doesn’t always tell the exact answer So we have to “guess” Human_T MatchHuman(float type){ float threashold = 0.2; type = type*HUMAN_TYPES; if(type HUMAN_TYPES+threashold) return NO_HUMAN; if(fabs(type-float(DRONE))<threashold) return DRONE; else if(fabs(type-float(MINE))<threashold) return MINE; else if(fabs(type-float(FARMER))<threashold) return FARMER; else if(fabs(type-float(INFANTRY))<threashold) return INFANTRY; else if(fabs(type-float(SETTLER))<threashold) return SETTLER; else if(fabs(type-float(ARMOR))<threashold) return ARMOR; else if(fabs(type-float(JAMMER))<threashold) return JAMMER; else if(fabs(type-float(ARTILLERY))<threashold) return ARTILLERY; return NO_HUMAN; } CNNPlayer::DoTurns(){ …… //normalize the output float low = actual[0]; float high = actual[0]; float avg = 0; for(i=0; i<OUTPUT_NEURONS; i++){ if(low>actual[i]) low=actual[i]; if(high<actual[i]) high=actual[i]; avg+=actual[i]/OUTPUT_NEURONS; } for(i=0; i<OUTPUT_NEURONS; i++){ if(actual[i]<=0) actual[i] = 0; else{ actual[i] = (actual[i]-low)/(high-low); //actual[i] = actual[i]/avg; } …… }

Science, Portland State University22 NN Outputs (4 hour & 1M inter.)

Science, Portland State University23 Recall the FSM

Science, Portland State University24 A Random NN Give You This!

Science, Portland State University25 Agenda About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion

Science, Portland State University26 Evaluations Test based on each turn –Only for each turn, the Chaos’ brain is certain –My AI performs well (win) if and only if: Given random initial treasury and map It has advantage against most Chaos’ brains (money earned) Good performance on each turn leads to final win

Science, Portland State University 27 Statistics of My FSM Initial Treasury We want this! Not well dealing with money between this range

Science, Portland State University 28 Statistics of My NN AI Initial Treasury We want this!

Science, Portland State University29 Agenda About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion

Science, Portland State University30 Conclusion Studied AP –Adaptive turn-based strategy game Static FSM –Benchmark –Train NN NN –Developed a demo –Trained by FSM –Need more time to train it

Science, Portland State University31 Discussion Static FSM –Efficient to code –Pros: do exactly what you want –Cons: do exactly what you want NN –Like dealing with a child Patient Be careful what you feed to it Try to understand a child’s language More trained, usually better performance

Science, Portland State University32 Discussion (Cont.) Coding & debugging issues –Error in coding NN is not easy to be detected Wrong training data Inconsistent between training inputs and using inputs –Be careful about important parts Training data Training process –Set breakpoint to check –Code review –Train it by the same example, see if it adapts as expected

Science, Portland State University33 Questions ?