Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions Sunju Park Management Science and Information Systems Department Rutgers Business School, Rutgers University Edmund H. Durfee Artificial Intelligence Laboratory, University of Michigan William P. Birmingham Math & Computer Science Department, Grove City College Presenter: TinTin Yu

Introduction Not like tradition auctions  Single seller and multiple buyers (e.g. eBay) Continuous Double Auctions (CDA)  Buyers place bids, and sellers place offers to the same items.  We have a match whenever a buyer’s bid is higher than a seller’s offer.  (e.g. Name your price (hotel.com?) Goal  To determine the optimal price/offer for a seller in order to gain the maximum profit.

Definitions Notation: bbss p  b: buyer’s bid; s: seller’s offer  s p : seller’s offer that was just submitted  bbss p : a queue in ascending order (of price) Clearing Price (CP)  bs p bs: When an offer is less than a bid  s p <=CP<=b (the right most b)  We use s p in this paper.

Definitions Markov Chains (Markov state machine)  Probabilistic finite state machine  Input is ignored  We uses first-order Markov chain only First-order means the probability of the present state is a function only to its direct predecessor states.

p-strategy Algorithm (1/2)

p-strategy Algorithm (2/2) Information used by p-strategy

Step1: Building Markov Chains (1/3) Given a current state (bbs).  When the p-seller (a seller use p-strategy) submit its offer s p, there are four possible next auction states.  We make these states the initial states of the Markov Chain.

Step1: Building Markov Chains (2/3) From the initial states, we keep populate the (bbss) queue by either submitting a new buyer bid or a seller offer. If we have a match, it goes to the SUCCESS state. If it goes out of the bound (maximum number of standing offers), it goes to the FAIL state.

Step1: Building Markov Chains (3/3) The MC model of the CDA with starting state (bbs) and the number of bids and offers are limited to 5 each.

Step2: Compute Utilities (1/5) Step2.1: The utilities function  P s (p): probability of success at price p  U(Payoff s (p)): utilities of payoff if the offer receives a match  CP: clearing price  C: cost  TD(  s/f ): delay overhead

Step2: Compute Utilities (2/5) Things we need to compute for each p

Step2: Compute Utilities (3/5) Step2.2.1: Transition Probabilities  Going from state (bbs) to (bbss p ) at time step n  That is P (bbss p | bbs);  Applying Baye’s rule;  Evaluating using probability density function (PDF), f(s); bababa…

Step2: Compute Utilities (4/5) Step2.2.2: TD(  s/f ): delay overhead  Too complex to cover in details  It involves building a transition probability matrix P from the states of the Markov Chain we built in step1.  Here is listed equations:   : reward = c (a constant) except for the initial states and the absorbing states   : the number of visits to state (…) until it goes to S.

Step2: Compute Utilities (5/5) Plug in the numbers and we will get a expected utility value associated with price p. The algorithm find the optimal price p by looping through all p in a possible range. Time complexity of the algorithm is O(  n 3 ), where  is the number of possible prices, n is the number of MC states.

Benchmark (1/6) Agents used for comparison  FM: Fixed-Markup bids its cost plus some predefined markup  RM: Random-Markup bids its cost plus some random markup  CP: Clearing-Price obtains a clearing-price quote (similar to FM agent)  OPT: Post-facto Optimal our benchmark strategy. Given it “knows” exactly everything about the future (no uncertainty at all), it returns the maximum profit an agent may have achieved.

Benchmark (2/6)

Benchmark (3/6): p-strategy vs other Results: Arrival rate:  0.4=high  0.1=low negotiation zone  narrow:  =5

Benchmark (4/6): p-strategy vs other Results: Arrival rate:  0.4=high  0.1=low negotiation zone  narrow:  =25

Benchmark (5/6) : p-strategy vs itself Results Profit of individual p-agent decrease as the number of p-agents increase. However, when there is more buyers, p-agents are able to gain similar profit at the expense of buyers.

Benchmark (6/6) : CP vs multiple p and CP Results CP-strategy agents are able to raises profit as the number mixed p-agents and CP-agents increase.

Conclusion Summary:  p-strategy is based on stochastic modeling of the auction process.  It works while it does not need to consider much about the other individual agents. Time complexity only depends on the number of MC states, not the number of agents.  It out performs other agents (FM/ RM/ CP) Future Work  Similar strategy can be apply to buyers.  Analysis shows an average of 20% gap between p-strategy and the optimal one.  Ongoing work: hybrid strategy. This adaptive approach allow the agent to figure out when to use stochastic model and when to use some simpler strategies.

Question to think about Human can think very differently:  e.g. Selling a 50” plasma HDTV Place a very low selling price like $1.00 without a hidden limit. Shipping cost = $ ?! Can artificial intelligent agents think outside the box?

Your Questions

Bibliography Park, S., Durfee, E.H. and Birmingham, W.P. (2004) "Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions", Volume 22, pages