2006 AAAI Computer Poker Competition Michael Littman Rutgers University Martin Zinkevich Christian Smith Luke Duguid U of Alberta.

Slides:

Advertisements

Similar presentations

Building Agents for the Lemonade Game Using a Cognitive Hierarchy Population Model Michael Wunder Michael Kaisers Michael Littman John Yaros.

Advertisements

Outline Background  What is Texas Hold'em?  What are Bayesian Networks?  What is BPP? Aims  Initial opponent model  Adaptive opponent model  Performance.

Evaluation Through Conflict Martin Zinkevich Yahoo! Inc.

Randomized Strategies and Temporal Difference Learning in Poker Michael Oder April 4, 2002 Advisor: Dr. David Mutchler.

A measurement of fairness game 1: A box contains 1red marble and 3 black marbles. Blindfolded, you select one marble. If you select the red marble, you.

Neural Networks for Opponent Modeling in Poker John Pym.

Use logic to teach the computer how to play a game

Created by Charles Jenkins

mike How fast can you identify all the shapes? See if you can beat the clock by identifying all the shapes before the time expires. Challenge yourself.

Horse race © Horse race: rules 1.Each player chooses a horse and puts it into a stall. Write your name next to the.

Ladder & Pyramid Tournaments. Ladder Structured like a step ladder Structured like a step ladder A player is placed on each rung of the ladder A player.

INFORMS 2006, Pittsburgh, November 8, 2006 © 2006 M. A. Zinkevich, AICML 1 Games, Optimization, and Online Algorithms Martin Zinkevich University of Alberta.

Andrew DePriest 4/12/2011 CSCE 390 – 001 Professional Issues in Computer Science and Engineering (Article found in the March 14, 2011 edition of ACM TechNews,

Poker for Fun and Profit (and intellectual challenge) Robert Holte Computing Science Dept. University of Alberta.

Probability And Expected Value ————————————

AWESOME: A General Multiagent Learning Algorithm that Converges in Self- Play and Learns a Best Response Against Stationary Opponents Vincent Conitzer.

Mechanism Design: Online Auction or Packet Scheduling Online auction of a reusable good (packet slots) Agents types: (arrival, departure, value) –Agents.

Presenter: Robert Holte. 2 Helping the world understand … and make informed decisions. * * Potential beneficiaries: commercial games companies, and their.

Online Chess Project 3 Due date: April 17 th. Introduction Third in series of three projects This project focuses on adding online support –2 players.

Better automated abstraction techniques for imperfect information games, with application to Texas Hold’em poker * Andrew Gilpin and Tuomas Sandholm, CMU,

Models of Strategic Deficiency and Poker Workflow Inference: What to do with One Example and no Semantics.

Introduction to the Big Stack Strategy (BSS) Strategy: No Limit.

Mean, Variance, and Standard Deviation

The Parameterized Poker Squares EAAI NSG Challenge

Designing games to facilitate learning How to design cheap and effective traditional games.

1.5 Conditional Probability 1. Conditional Probability 2. The multiplication rule 3. Partition Theorem 4. Bayes’ Rule.

Bootstrapping (And other statistical trickery). Reminder Of What We Do In Statistics Null Hypothesis Statistical Test Logic – Assume that the “no effect”

Poker Download A most popular card game or group of card games is called poker. Players compete against one another by betting on the values of each player's.

Ankur Patankar GT ID:

Expected values and variances. Formula For a discrete random variable X and pmf p(X): Expected value: Variance: Alternate formula for variance:  Var(x)=E(X^2)-[E(X)]^2.

Chapter 5.1 Probability Distributions.  A variable is defined as a characteristic or attribute that can assume different values.  Recall that a variable.

SARTRE: System Overview A Case-Based Agent for Two-Player Texas Hold'em Jonathan Rubin & Ian Watson University of Auckland Game AI Group

Quark Card Game by Helio Takai (and dice game)

Introduction to Chess Club

DISCRETE PROBABILITY DISTRIBUTIONS

Binomial Experiment A binomial experiment (also known as a Bernoulli trial) is a statistical experiment that has the following properties:

Games. Adversaries Consider the process of reasoning when an adversary is trying to defeat our efforts In game playing situations one searches down the.

The EAR PULL North Pole High School North Pole, Alaska Mr. Richard Henert By: Alizey A. Amber L. Michael S. Edward M. Meghan G.

Chapter 5 Discrete Probability Distributions. Introduction Many decisions in real-life situations are made by assigning probabilities to all possible.

Neural Network Implementation of Poker AI

The pétanque is a very popular sport in Corsica. Everybody can play, there are usually competitions and we can win money or it’s just for fun. Pétanque.

Hypothesis Testing for Standard Deviation or Variance.

Lecture 12. Game theory So far we discussed: roulette and blackjack Roulette: – Outcomes completely independent and random – Very little strategy (even.

Steering Evolution and Biological Adaptation Strategically: Computational Game Theory and Opponent Exploitation for Treatment Planning, Drug Design, and.

From Pure Math to Data Science: My Perspective Paul Raff Principal Data Scientist, Analysis and Experimentation, Microsoft October 1, 2015 My

English and American sports Презентацию выполнила Головкова А.Е., учитель МБОУ «Майская СОШ»

Science Ch. 1. Choose Your Contestant You are about to face a “ mob ” of ___ opponents in a winner takes all quiz challenge. Wrong answers from the mob.

 Sport plays a significant part in the life of many Czechs who are generally loyal supporters of their favourite teams or individuals.  The two leading.

Artificial Neural Networks And Texas Hold’em ECE 539 Final Project December 19, 2003 Andy Schultz.

A nontransitive talk James Grime. Hi Jim, I was wondering if you can design a set of five nontransitive dice.

A Game AI Coding Competition Pavel Dionisev & Nicolas Décrevel CERN

Defending When Outnumbered How to defend when your opponents counter attack.

In games of chance the expectations can be thought of as the average outcome if the game was repeated multiple times. Expectation These calculated expectations.

OPPONENT EXPLOITATION Tuomas Sandholm. Traditionally two approaches to tackling games Game theory approach (abstraction+equilibrium finding) –Safe in.

CONCEPT OF HEADWAY- START(HANDICAPPING) IN THE GAME OF CRICKET Presented By Raof Ahmad Bhat Research Scholar Department of Physical Education, Aligarh.

John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center with a lot of slides from Tuomas Sandholm Copyright 2015 Poker.

Implemented: Davidov Dmitri Scientific leadership: Steblina I. B.

Stochastic tree search and stochastic games

Joint work with Sam Ganzfried

Game Theory Just last week:

Ashley Cripps1, Luke Hopper1,2, Christopher Joyce1

Noam Brown and Tuomas Sandholm Computer Science Department

The Parameterized Poker Squares EAAI NSG Challenge

Strategies for Poker AI player

Tobias Mahlmann and Mike Preuss

Warm Up Imagine a family has three children. 1) What is the probability the family has: 3 girls and 0 boys 2 girls and 1 boy 1 girl and 2 boys 0 girls.

Big problem  small steps

Term Project: Poker Game

Expected values and variances

Presentation transcript:

2006 AAAI Computer Poker Competition Michael Littman Rutgers University Martin Zinkevich Christian Smith Luke Duguid U of Alberta

What is Poker? The OR View –A partial information game with over states. The AI View –A huge opponent modeling challenge. The Public View –Really popular! Lots of fun!

What is the Game? Play 1000 hands of Heads-up Limit Texas Hold’em Poker against an opponent Reset bots, switch seats, and play again with the same hands. –lower variance –more fair Repeat 6-20 times –Can treat the outcome of the duplicate match as a random variable which we have sampled several times.

How do you win? Make money! Bankroll Competition –Against overall, have the maximum total bankroll –Highlights opponent modeling and learning Series Competition –Against individuals, have positive bankroll –Highlights the OR problem of “solving” the game

The Teams Hyperborean: University of Alberta Edmonton, Canada Bluffbot: Finland Monash: Monash University, Australia Teddy: USA Gs2: Carnegie Mellon University, Pittsburgh, USA

Before I Begin There is a period that is allocated for competitors to contest the results that has yet to expire.

Bankroll Results CompetitorWinnings (small bets/hand) Teddy (USA) sb/h

Bankroll Results CompetitorWinnings (small bets/hand) Monash (Monash U., Australia) sb/h Teddy (USA) sb/h

Bankroll Results CompetitorWinnings (small bets/hand) Bluffbot (Finland) sb/h Monash (Monash U., Australia) sb/h Teddy (USA) sb/h

Bankroll Results CompetitorWinnings (small bets/hand) Hyperborean (U. Alberta) sb/h Bluffbot (Finland) sb/h Monash (Monash U., Australia) sb/h Teddy (USA) sb/h

Bankroll-Heads Up Hyperborean (U Alberta) Bluffbot (Finland) Monash (Monash U) Teddy (USA) Hyperborean (U Alberta) Hyperborean Wins Hyperborean Wins Hyperborean Wins Bluffbot (Finland) Hyperborean Wins Bluffbot WinsTeddy Wins Monash (Monash U) Hyperborean Wins Bluffbot WinsMonash Wins Teddy (USA) Hyperborean Wins Teddy WinsMonash Wins

Bankroll-Heads Up (Small Bets/Hand) Hyperborean (U Alberta) Bluffbot (Finland) Monash (Monash U) Teddy (USA) Hyperborean (U Alberta) ± ± ± Bluffbot (Finland) ± ± ± Monash (Monash U) ± ± ± Teddy (USA) ± ± ±

Bankroll-Overall Significance Difference between Bluffbot and U of Alberta small bets/hand standard deviations

The Most Interesting Result In the bankroll competition, in head-to- head, BluffBot beat Monash who beat Teddy who beat BluffBot A practical example of the non-transitivity of poker

Results-Series (Small Bets/Hand) Hyperborean (U Alberta) Bluffbot (Finland) Gs2 (CMU) Monash (Monash U) Hyperborean (U Alberta) Winner: Hyperborean Winner: Hyperborean Winner: Hyperborean Bluffbot (Finland) Winner: Hyperborean Winner: Bluffbot Winner: Bluffbot Gs2 (CMU) Winner: Hyperborean Winner: Bluffbot Winner: Gs2 Monash (Monash U) Winner: Hyperborean Winner: Bluffbot Winner: Gs2

Results-Series (Small Bets/Hand) Hyperborean (U Alberta) Bluffbot (Finland) Gs2 (CMU) Monash (Monash U) Hyperborean (U Alberta) ± ± ±0.029 Bluffbot (Finland) ± ± ± Gs2 (CMU) ± ± ± Monash (Monash U) ± ± ±0.0398

Things to Never Assume All bots will work on the competition machines the first time The server code is bug-free Everybody has a common idea of the rules of poker (or even heads-up Texas Hold’em) People can write code instantaneously

Exhibitionary Aspects of the Competition Bots submitted late Bots debugged after the deadline The time limit was very large for the series competition

Maybe Next Year More advance notice Competitors need access to one of the machines they will use Server code needs to be frozen before the competition begins More variants of poker need to be included, especially: –>2 players –>1000 hands There has to be a 7 sec/hand time limit

Is Poker “Solved”? No one has ever solved a four-round abstraction of poker without a partition into the early game and the late game. The game of poker is also about your opponent. For instance, playing rock- paper-scissors against a four-year-old is different than against an adult.

Next Year This summer: write the rules This winter: write the code Next summer: next competition

Are You Interested? Come talk to us

Summary The poker competition brought together five teams in two competitions. The competition was very close, and very interesting, non-transitive (A beats B beats C beats A) performance was observed. A freeware codebase was developed for future competitions.