Optimal Electricity Supply Bidding by Markov Decision Process Authors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert Dahlgren Presentation Review By: Feng Gao, Esteban Gil, & Kory Hedman IE 513 Analysis of Stochastic Systems Professor Sarah Ryan April 28, 2005

Outline Summary of the previous presentation Numerical Simulation Case Study Example 1 – Production Limit Example 2 – Market Power Conclusion Extension

Summary of previous presentation Solution Procedure Value Iteration Model Validation Accumulate actual data and observations from the market over a period of time Estimated rewards r(i,j,a) and actual rewards w(i,j,a) are analyzed by linear regression.

Summary of previous presentation 3 suppliers: GenCoA, GenCoB, and GenCoC GenCoA is the decision maker using the Markov Decision Process technique GenCoA: 1 generating unit GenCoB: 2 generating units GenCoC: 2 generating units Planning Horizon: 7 days (bid decision for next day considers the entire week ahead)

Summary of previous presentation GenCoA makes a decision from a set of pre-specified decision options GenCoA models GenCoB /C ‘s bidding behavior by prices, quantities and the associated probabilities based on his knowledge and information

Comparison of Two Examples With production limit, bidder is concerned with saving resources for future more expensive periods (example 1) With market power, bidder is concerned with properly influencing the future spot price to maximize future profit (example 2) Knowing whether the bidder has market power/ production limit or not is crucial since the relationship between profits and decisions/resource depend on each other

Example 1 Decision-maker has a production limit over the planning horizon 210 states, 10 possible production levels between 0 and the production limit The parameters for state 1 are listed in Table III

Example 1 GenCoA’ s Cost, Production Limit and Decision Options GenCoB’s Decision Options in State 1

Example 1 Value iteration algorithm is used to calculate the optimal decision for each state and each stage

Example 1 One rational strategy, Peak-Load Strategy, is chosen for comparison with the BIDS results Peak-Load Strategy: the decision maker bids at the cost, divides the production limit evenly over the week and sells only at the peak load period on each day. The result shows that the expected rewards from BIDS are consistently higher than that of the Peak-Load Strategy. Optimal strategy is time dependent due to the production limit In some states the optimal decision is not to sell, but to save the resources for more profitable days

Example 2 Decision-maker has market power: it can manipulate the bid to influence the spot price Decision-maker has no production limit Decision-maker makes the bidding decision to maximize the expected reward over the planning horizon There are 21 states in the MDP model, since no production limit Daily Maximum Strategy and BIDS results are used for comparison

Example 2 Daily Maximum Strategy maximizes the daily reward without considering how the bid affects the market trend

Example 2 Results Daily maximum strategy is time independent: decision-maker makes the same decision as long as the system is in the same state BIDS results are time dependent: it takes into account how current bidding affects future spot prices

Example 2 Since the decision-maker has market power: Bidding at high prices gives the decision-maker a high immediate reward and the system transfers to high price states with higher probabilities Bidding at low prices gives the decision-maker a lower immediate reward and the system transfers to low price states with higher probabilities Decision-maker should not rely on the daily profit in making decisions

Conclusion The spot market bidding decision–making problem is formulated by a Markov Decision Process An algorithm is developed to calculate the transition probabilities and reward function Value iteration method is applied to solve the MDP Implementation and case study Three GenCos, GenCo A is the Decision Maker 5 Generators among the 3 GenCos Description of 2 Examples: Production Limit without Market Power Market Power without Production Limit

Extension Risk-neutral decision–maker is assumed. In order to incorporate risk attitude, the variance of decision– maker’s profits should be brought in the bidding decisions. A slightly different formulation, risk– sensitive MDP, can be used to solve this problem. The objective function is to maximize the total expected future reward during 7-day planning horizon. The 7-day planning horizon is chosen based on the accuracy of the short–term load forecasting. However, an optimal planning horizon should be discussed in the future. The competitors’ strategies are modeled as price, quantity and probability from the decision–maker ‘s point of view. However, game theory should be incorporated to model how competitors’ activities are affected by decision–maker’s decisions as well.

Questions???