Computational Intelligence in Games: An Overview Zahid Halim Faculty of Computer Science and Engineering Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi.

Layout What is AI/CI and ML Why Computer Games? How CI helps computer Games? Some Examples Key venues to publish work Future directions 12/19/2012Computational Intelligence in Games: An Overview2

AI vs. CI vs. ML Artificial Intelligence (Think like human, learn from experience, recognize patterns, make complex decisions based on knowledge and reasoning) – Machine learning – Knowledge representation – Natural Language Processing – Planning Robotics etc. Machine learning – Branch of AI – Construction and study of systems that can learn from data – messages to learn to distinguish between spam and non-spam messages – There is difference between ML and Data Mining too Computational Intelligence ( – Integrating the fields Artificial Neural Networks Evolutionary Computation Fuzzy Logic Computational Intelligence in Games: An Overview312/19/2012

They are related… But they are all different… I hope all of us understand difference between hard and soft computing AI CIML Computational Intelligence in Games: An Overview412/19/2012

Why Computer Games? 49% of U.S. households own a dedicated game console The average game player age is: 30 years Computational Intelligence in Games: An Overview512/19/2012

Why Computer Games? 42% of game players believe that computer and video games give them the most value for their money, compared with DVDs, music or going out to the movies Gamers who are playing more video games than they did three years ago are spending less time: – 59% playing board games – 50% going to the movies – 47% watching TV – 47% watching movies at home 62% of gamers play games with others, either in-person or online 78% of gamers who play with others do so at least one hour per week Computational Intelligence in Games: An Overview612/19/2012

Money Matters! Total: $24.75 Billion Computational Intelligence in Games: An Overview712/19/2012 But its not every thing!

What can Computational Intelligence do? Generate complete game Creation of intelligent game characters Creation of entertaining game characters Generating tracks for racing games. Adaptable player experience. Levels for action games. Generating maps for games. Computational Intelligence in Games: An Overview812/19/2012

Procedural Content Generation Lindenmayer system: A variant of a formal grammar, most famously used to model the growth processes of plant. Consists of: – An alphabet of symbols that can be used to make strings – A collection of production rules which expand each symbol into some larger string of symbols – An initial "axiom" string from which to begin construction – A mechanism for translating the generated strings into geometric structures. PCG can also generate weapons that player might require in a game Search based PCG is different Computational Intelligence in Games: An Overview912/19/2012

Some of the PCG based Games GameContentYear ToeJam & EarlThe random levels were procedurally generated.1991 The Elder Scrolls III: Morrowind Water effects are generated on the fly."Water Interaction" demo.2002 RoboBlitzXBox360 live arcade and PC2006 BorderlandsWeapons were generated depending upon the levels2009 Terraria 2D landscape was generated that a player can travel around.2011 Computational Intelligence in Games: An Overview1012/19/2012

Automated Game “entertaining” Generation Search Space Dimension Possible Values Select Values Checkers Chess Play Area Only black squares are used Both white & black squares are used Types of PiecesInitially 1, maximum 266 Number of pieces/type 12, variable (but max. 12) 16variable but at maximum 24 Initial position Black squares of first 3 rows Both white & black squares of first 2 rows Both white & black squares of first 3 rows Movement direction Diagonal forward and Diagonal, forward backward All directions, straight forward, straight forward and backward, L shaped, diagonal forward Step SizeOne Step One Step, Multiple Steps Capturing LogicStep overStep intoStep over, step into Game ending logic No moves possible for a player No moves possible for the king No moves possible for a player, no moves possible for the king Conversion LogicCheckers into king Soldiers into queen or any piece of choice Depends upon rules of the game Mandatory to captureYesNo Depends upon rules of the game Turn passing allowedNo Computational Intelligence in Games: An Overview1112/19/2012

Objective Function 1+1 Evolutionary Strategy (ES) 10 chromosomes are randomly initialized The evolutionary algorithm is run for 100 iterations Mutation only with probability of 30 percent One parent produce one child – Fitness difference is calculated – If it is greater than 4 (at least half times better) child is promoted to the next population Computational Intelligence in Games: An Overview1212/19/2012

Making Racing Fun Through Player Modelling and Track Evolution We have one or several car racing tracks with – Walls, Waypoints, Staring position of the car Car consist of – Sensor model to sense the environment – Discrete set of control commands Objective of the game is to pass as many waypoints in given timesteps. Car has 6 sensors, Speed of the car and Angle to the next waypoint Fully connected feedforward nets (MLPs) with the tanh transfer function. Only the weights of the networks are changed by evolution or back propagation Nine inputs (sensors and a bias input), Six hidden neurons Two output neurons are used. – The First output is interpreted as driving command – Second as steering command. Computational Intelligence in Games: An Overview1312/19/2012

Learning Behaviour: Backpropagation Human player drove a number of laps around a track, while the inputs from sensors and actions taken by the human were logged at each timestep. This log was then used to train a neural network controller to associate sensor inputs with actions using a standard backpropagation algorithm. Several variations on this idea were tried with very little success. Training often achieved low error rates (typically 0.05), none of the trained networks managed to complete even half a lap. A small amount of noise that is applied to sensors guarantees that the car does not simply replay the human action. Computational Intelligence in Games: An Overview1412/19/2012

Evolving Neural Network Agents in the NERO Video Game –real-time NeuroEvolution of Augmenting Topologies (rt-NEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played. –rtNEAT makes possible a new genre of video games in which the player teaches a team of agents through a series of customized training exercises. –In NEAT, the population is replaced at each generation. Everyone’s behaviour would change at once. Behaviours would remain static during the large gaps between generations –In rtNEAT, a single individual is replaced very few game ticks Computational Intelligence in Games: An Overview1512/19/2012

Conferences and Journals IEEE Computational Intelligence and Games IEEE Transactions on Computational Intelligence and AI in Games (IF 1.8) International Journal of Computer Games Technology International Conference on Computer Games (CGAMES) CGamesUSA International Conference on Computer Games Computational Intelligence in Games: An Overview1612/19/2012

Where are the opportunities? CIG for health care CIG for education Neuro Computer interface for games Physicological study via games Computational Intelligence in Games: An Overview1712/19/2012

Thanks for your patience Presentation available at:

Bibliography Halim, Zahid, A. Rauf Baig, and Hasan Mujtaba. "Measuring entertainment and automatic generation of entertaining games." International Journal of Information Technology, Communications and Convergence 1.1 (2010): Halim, Zahid, A. Rauf Baig, and Mujtaba Hasan. "Evolutionary Search For Entertainment In Computer Games." Intelligent Automation & Soft Computing 18.1 (2012): Halim, Zahid, and A. Raif Baig. "Evolutionary Algorithms towards Generating Entertaining Games." Next Generation Data Technologies for Collective Computational Intelligence. Springer Berlin Heidelberg, ESA 2012 Sales, Demographic and Usage Data Evolving Neural Network Agents in the NERO Video Game, Stanley et. al Acquiring Visibly Intelligent Behavior with Example-Guided Neuroevolution, Bryant et. al. Making Racing Fun Through Player Modeling, Togelius et. al. Evolutionary Search for Entertainment in Games, Halim et. al. 12/19/2012Computational Intelligence in Games: An Overview19