Optimal Electricity Supply Bidding by Markov Decision Process Authors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert Dahlgren Presentation Review By: Feng Gao, Esteban Gil, & Kory Hedman IE 513 Analysis of Stochastic Systems Professor Sarah Ryan February 14, 2005

Outline Introduction Purpose Problem Formulation Model Overview Summary

Introduction Electric Industry has Transitioned from Regulated to Deregulated Regulated: Vertically Integrated, Monopolistic Market Deregulated: Ideally, Perfect Competition Market Decision Analysis: Based on Profit & Competition

Introduction Cont’d Traditional Power System Generation, Transmission, Distribution, Consumption

Introduction Cont’d Structure of Power Market Optimize Resources With Competition

Introduction Cont’d Day Ahead Market Considered Inelastic Demand Generation Companies (GenCos) are Risk-Neutral GenCos Bid a Price (P) & Quantity (Q) Bids are Chosen from Cheapest to Most Expensive Market Clearing Price: P from the Highest Chosen Q (Similar to the New Zealand & Great Britain Electricity Markets)

Purpose Overall Objective: Maximize Expected Profit over a Planning Horizon of 7 Days For all States, Determine Optimal Bidding Strategy. Depends on: Competitors’ Bidding Load Forecasting Remaining Time Horizon (given day i, 7 – i) Production Limit (Max Available Supply Remaining over Planning Horizon) Accumulated Data over Time (Past Load & Price Data)

Problem Formulation States are Defined by 7 Variables: Peak Load & Peak Price (2) Off-Peak Load & Off-Peak Price (2) Current Production Limit for the Remaining Planning Horizon (1) Load Forecast for the Following Day (2) Aggregation Limits Number of States P & D Broken into High, Medium, & Low Illogical States Ignored: (High P & Low Q, etc.)

Problem Formulation Cont’d Transition Probability Depends on: Current State i, Subsequent State j, Decision a Pr (i, j, a) Decision Maker Receives Reward R (i, j, a) State of the Market Defines the Competitors’ Bids & Decision Maker’s Bidding Options Bid Prices are Determined Using a Staircase Supply Fn for Varying MW

Problem Formulation Cont’d MDP Algorithm Considers: Rewards Based on Load Forecast Decisions of a State Competitors’ Bidding Characteristics Decision Options Affect Transition Probabilities & Rewards Competitors’ Bids are Independent Scenarios (s) are exclusive

Model Overview Probability of Scenario s: Pr (i, n, k): Probability that Supplier n (n ~= m) Chooses Option k in State i m is decision maker Competitors’ Bids are Independent Remaining Production Limit: q (i, s, t) is the Q used in period t for scenario s. Spot Price for Scenario s: SP(i, s, t)

Model Overview Cont’d Probability to Move from State i to j: Pr (i, LF (j, t) = Probability that Load Forecast for Day After Tomorrow is LF (j, t) Given Present State i Reward for Decision Maker, r (i, s):

Model Overview Cont’d Reward for Transition from i to j & Decision A is Sum of Rewards Weighted by Conditional Probabilities: V (i, T+1): Total Expected Reward in T+1 Remaining Stages from State i Solved by Value Iteration

Summary Introduction Purpose Problem Formulation Model Overview Electric Market is now Competitive GenCos Bid on Demand Purpose MDP Used to Determine Optimal Bidding Strategy Problem Formulation Transition Probability Determined by Current State, Subsequent State, & Decision Made 7 Variables to Define a State Aggregation Used to Limit Dimensionality Problems Model Overview 7 Day Planning Horizon Objective is to Maximize Summation of Expected Reward Value Iteration

Questions???

References Song, H.; Liu, C.-C.; Lawarree, J.; Dahlgren, R.W, “Optimal Electricity Supply Bidding by Markov Decision Process,” IEEE Transactions. Power Systems, Vol. 15, no. 2, pp 618-624, May 2000. http://www.acclaimimages.com/_gallery/_pages/0037-0409-0607-4216.html