1. Optimal Electricity Supply Bidding by Markov Decision Process
Authors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert Dahlgren
Presentation Review By:
Feng Gao, Esteban Gil, & Kory Hedman
IE 513 Analysis of Stochastic Systems
Professor Sarah Ryan
February 14, 2005

2. Outline Introduction Purpose Problem Formulation Model Overview
Summary

3. Introduction
Electric Industry has Transitioned from Regulated to Deregulated
Regulated: Vertically Integrated, Monopolistic Market
Deregulated: Ideally, Perfect Competition Market
Decision Analysis: Based on Profit & Competition

4. Introduction Cont’d Traditional Power System
Generation, Transmission, Distribution, Consumption

5. Introduction Cont’d Structure of Power Market
Optimize Resources With Competition

6. Introduction Cont’d Day Ahead Market Considered Inelastic Demand
Generation Companies (GenCos) are Risk-Neutral
GenCos Bid a Price (P) & Quantity (Q)
Bids are Chosen from Cheapest to Most Expensive
Market Clearing Price: P from the Highest Chosen Q
(Similar to the New Zealand & Great Britain Electricity Markets)

7. Purpose
Overall Objective: Maximize Expected Profit over a Planning Horizon of 7 Days
For all States, Determine Optimal Bidding Strategy. Depends on:
Competitors’ Bidding
Load Forecasting
Remaining Time Horizon (given day i, 7 – i)
Production Limit (Max Available Supply Remaining over Planning Horizon)
Accumulated Data over Time (Past Load & Price Data)

8. Problem Formulation States are Defined by 7 Variables:
Peak Load & Peak Price (2)
Off-Peak Load & Off-Peak Price (2)
Current Production Limit for the Remaining Planning Horizon (1)
Load Forecast for the Following Day (2)
Aggregation Limits Number of States
P & D Broken into High, Medium, & Low
Illogical States Ignored: (High P & Low Q, etc.)

9. Problem Formulation Cont’d
Transition Probability Depends on:
Current State i, Subsequent State j, Decision a
Pr (i, j, a)
Decision Maker Receives Reward
R (i, j, a)
State of the Market Defines the Competitors’ Bids & Decision Maker’s Bidding Options
Bid Prices are Determined Using a Staircase Supply Fn for Varying MW

10. Problem Formulation Cont’d
MDP Algorithm Considers:
Rewards Based on Load Forecast
Decisions of a State
Competitors’ Bidding Characteristics
Decision Options Affect Transition Probabilities & Rewards
Competitors’ Bids are Independent
Scenarios (s) are exclusive

11. Model Overview Probability of Scenario s:
Pr (i, n, k): Probability that Supplier n (n ~= m) Chooses Option k in State i
m is decision maker
Competitors’ Bids are Independent
Remaining Production Limit:
q (i, s, t) is the Q used in period t for scenario s.
Spot Price for Scenario s: SP(i, s, t)

12. Model Overview Cont’d Probability to Move from State i to j:
Pr (i, LF (j, t) = Probability that Load Forecast for Day After Tomorrow is LF (j, t) Given Present State i
Reward for Decision Maker, r (i, s):

13. Model Overview Cont’d
Reward for Transition from i to j & Decision A is Sum of Rewards Weighted by Conditional Probabilities:
V (i, T+1): Total Expected Reward in T+1 Remaining Stages from State i
Solved by Value Iteration

14. Summary Introduction Purpose Problem Formulation Model Overview
Electric Market is now Competitive
GenCos Bid on Demand
Purpose
MDP Used to Determine Optimal Bidding Strategy
Problem Formulation
Transition Probability Determined by Current State, Subsequent State, & Decision Made
7 Variables to Define a State
Aggregation Used to Limit Dimensionality Problems
Model Overview
7 Day Planning Horizon
Objective is to Maximize Summation of Expected Reward
Value Iteration

15. Questions???

16. References
Song, H.; Liu, C.-C.; Lawarree, J.; Dahlgren, R.W, “Optimal Electricity Supply Bidding by Markov Decision Process,” IEEE Transactions. Power Systems, Vol. 15, no. 2, pp , May 2000.