8/8/2019 PokerBots2008[2]

1/21

POKER BOTS 2008

Jeff Berkowitz

pdxjjb@gmail.com

8/8/2019 PokerBots2008[2]

2/21

Tonights Talk

Why is poker an interesting problem?

A bit about poker strategy

A simple tournament bot

A more interesting tournament bot

Onward and beerward

8/8/2019 PokerBots2008[2]

3/21

Before we begin a word about tells

Somebody always asks about Poker tells

Computers cant read tells! (etc.)

Answer: tells are critical to success if youre making a movie.

Seriously: tells are not a big part of poker.

Questions about tells usually come from people

who havent yet grasped what a deep intellectual

challenge the game is.

8/8/2019 PokerBots2008[2]

4/21

Characterizing Games

Open

Information

Limited

Information

Deterministic

Nondeterministic

Chess

Checkers

Go

Battleship

Minesweeper

Backgammon

Poker

Stock market

8/8/2019 PokerBots2008[2]

5/21

Strategy (1) Equity Basics

You hold: 5i 4i

Board shows: Jj Tk 6l 2i

You believe your opponent has JJ or TT

She bets. Should you call?

It depends on the pot odds

4 of the 46 unseen cards give you a likely winner Pot odds must be 11 : 1 (or so) to call

Limit Poker

8/8/2019 PokerBots2008[2]

6/21

Strategy (2) Controlling Equity

You Hold: Kj Tk

Board shows: Kk 7k 5j

Pot contains 8 bets [note: small bets]

You bet your pair of kings [9 bets in pot]

Turns out, player on your left holds 9i 8i

Your bet helped give him the odds to call You should have checked, hoping to raise

Faced your opponent with calling 2 bets

Limit Poker

8/8/2019 PokerBots2008[2]

7/21

Deception (1) Face Up Poker

Imagine Holdem with hold cards face-up

Well say heads-up (2 players) for simplicity

All equity decisions are certain Your turn: simulate a few million hands

< 1 second on modern CPUs

Model opponents as equally perfect

Bet based on the simulation outcome

Limit Poker

8/8/2019 PokerBots2008[2]

8/21

Deception (2) Your Hidden Cards

Now imagine you get to hide hole cards

But the other player still plays face-up ;-)

You play the same optimal strategy And your opponent knows it (important)

Your bet positive expectation

Negative for them so they will always fold

So you mustsandbag some of your hands

Now defining optimal is harder

Limit Poker

8/8/2019 PokerBots2008[2]

9/21

NL Tourney Strategy (1) - Background

In general, for both limit and no-limit

Pre-flop play is simpler, more formulaic

Because there is less information to go on

Post-flop play is harder

More information available

More time (flop, turn, river)

More room for deception, application of skill,

8/8/2019 PokerBots2008[2]

10/21

NL Tourney Strategy (2)

But in no-limit, post-flop play is optional

All-in on the pre-flop no post-flop decisions!

Creates worst possible equity position for others Only possible in no-limit

Especially important [later] in tournaments

Advance of the blinds

Consequence: Move-in or All-in players

Book Kill Phil [Rodman & Nelson 2005]

8/8/2019 PokerBots2008[2]

11/21

Digression: Poker in Academia

University of Alberta Poker Research Group

Their work started long before the fad 1995

Numerous papers on the internet www.poker.cs.ualberta.ca

Excellent software (100% pure Java)

Spinout company Poker Academy

The company is actually called Biotools

Product has a plug-in API for bots, Meerkat

8/8/2019 PokerBots2008[2]

12/21

Demo - KillPhilBot

Implements Kill Phil Beginner strategy

From ibid (Kill Phil, Rodman & Nelson)

Modified with a slightly smarter postflop

8/8/2019 PokerBots2008[2]

13/21

Rules, rules, lotsa rules

A nontrivial rule-based bot gets hairy

If this else if this then that else if the other thing

Tracing scattered throughout Must restart Poker Academy to change the rules

Solution: separate behaviors from code

Rule (not really) interpretation system

In the future, every programmer will have their

own language (me, 2008)

Poker Rules Definition Language - PRDL

8/8/2019 PokerBots2008[2]

14/21

PRDLBot (Phil Purdle)

Express decision tree in PRDL

Parse with JJTree / javacc

Evaluate the parse tree for the action No code generation (although there could be)

Advantages:

Put tracing in the engine not the rules definition

Change and reload without restarting PA

Not fully implemented yet

8/8/2019 PokerBots2008[2]

15/21

PRDL Example (1)

stage = PREFLOP // all identifiers refer back to bot core

{

m > 30 // when we have a huge stack

{

holeCards in [KPGROUP_1, QQ, KQs] // use of a list of expressions

{

toCall

8/8/2019 PokerBots2008[2]

16/21

PRDL Example (2)

: // otherwise, raised pot

{

numberOfRaises = 1

{

action [BET, max[5 * raise, bankroll / 6]]

}

: // multiple raisers

{

action [BET, bankroll]

}

}

}

: // otherwise big stack, we don't have ultra-strong cards: we're done

{ action CHECK_FOLD // use of an expression that is not a list

}

}

}

: // Post-flop

{

// . . . . . etc . . . . .}

8/8/2019 PokerBots2008[2]

17/21

PRDLBot Tour

8/8/2019 PokerBots2008[2]

18/21

Bibliography (1 of2)

U of Alberta (CA) Poker Research Group:

http://poker.cs.ualberta.ca/

Scroll to Publications. Recommended there:

http://poker.cs.ualberta.ca/papers/AIJ02.html

Great online bot programming articles:

http://www.codingthewheel.com/category/poker

Poker Academy software: http://www.poker-academy.com/

http://forums.poker-academy.com/viewforum.php?f=3

8/8/2019 PokerBots2008[2]

19/21

Bibliography (2 of2)

Best overall poker web site / book publishers:

http://www.twoplustwo.com/

Recommended books from them: See http://twoplustwo.com/books.php

Best for beginners and for learning basic strategy are

Ed Millers books, Gettings Startedin Holdemand

Small Stakes Holdem: Winning Big with Expert Play. Best for learning tournament play are the Harrington

and Robertie book series, Harrington on Holdem

(volumes 1 through 3, to be studied in order).

8/8/2019 PokerBots2008[2]

20/21

Online Poker

Early years (1990s 2002)

IRC, academic, first commercial sites

Fad (2003 2006) TVwith exposed hole cards, Chris Moneymaker

Reality (2006 - )

UIGEA, bot suspicions

No individual has ever been arrested or

charged with any crime for playing poker

online in the United States.

8/8/2019 PokerBots2008[2]

21/21

Cheating at Online Poker

General approach

Write bot to Meerkat interfaces

Test and debug to high level of skill in PA Emulate Meerkat on a commercial poker client

Consider Omaha

4 hole cards, must play 2

Like playing 6 Holdem hands simultaneously

Often played for a split high/low pot (12 hands)

Computer advantage is maximized