1. Optimal Electricity Supply Bidding by Markov Decision Process
Authors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert Dahlgren
Presentation Review By:
Feng Gao, Esteban Gil, & Kory Hedman
IE 513 Analysis of Stochastic Systems
Professor Sarah Ryan
April 28, 2005

2. Outline Summary of the previous presentation
Numerical Simulation Case Study
Example 1 – Production Limit
Example 2 – Market Power
Conclusion
Extension

3. Summary of previous presentation
Solution Procedure
Value Iteration
Model Validation
Accumulate actual data and observations from the market over a period of time
Estimated rewards r(i,j,a) and actual rewards w(i,j,a) are analyzed by linear regression.

4. Summary of previous presentation
3 suppliers: GenCoA, GenCoB, and GenCoC
GenCoA is the decision maker using the Markov Decision Process technique
GenCoA: 1 generating unit
GenCoB: 2 generating units
GenCoC: 2 generating units
Planning Horizon: 7 days (bid decision for next day considers the entire week ahead)

5. Summary of previous presentation
GenCoA makes a decision from a set of pre-specified decision options
GenCoA models GenCoB /C ‘s bidding behavior by prices, quantities and the associated probabilities based on his knowledge and information

6. Comparison of Two Examples
With production limit, bidder is concerned with saving resources for future more expensive periods (example 1)
With market power, bidder is concerned with properly influencing the future spot price to maximize future profit (example 2)
Knowing whether the bidder has market power/ production limit or not is crucial since the relationship between profits and decisions/resource depend on each other

7. Example 1
Decision-maker has a production limit over the planning horizon
210 states, 10 possible production levels between 0 and the production limit
The parameters for state 1 are listed in Table III

8. Example 1 GenCoA’ s Cost, Production Limit and Decision Options
GenCoB’s Decision Options in State 1

9. Example 1
Value iteration algorithm is used to calculate the optimal decision for each state and each stage

10. Example 1
One rational strategy, Peak-Load Strategy, is chosen for comparison with the BIDS results
Peak-Load Strategy: the decision maker bids at the cost, divides the production limit evenly over the week and sells only at the peak load period on each day.
The result shows that the expected rewards from BIDS are consistently higher than that of the Peak-Load Strategy.
Optimal strategy is time dependent due to the production limit
In some states the optimal decision is not to sell, but to save the resources for more profitable days

11. Example 2
Decision-maker has market power: it can manipulate the bid to influence the spot price
Decision-maker has no production limit
Decision-maker makes the bidding decision to maximize the expected reward over the planning horizon
There are 21 states in the MDP model, since no production limit
Daily Maximum Strategy and BIDS results are used for comparison

12. Example 2
Daily Maximum Strategy maximizes the daily reward without considering how the bid affects the market trend

13. Example 2 Results
Daily maximum strategy is time independent: decision-maker makes the same decision as long as the system is in the same state
BIDS results are time dependent: it takes into account how current bidding affects future spot prices

14. Example 2 Since the decision-maker has market power:
Bidding at high prices gives the decision-maker a high immediate reward and the system transfers to high price states with higher probabilities
Bidding at low prices gives the decision-maker a lower immediate reward and the system transfers to low price states with higher probabilities
Decision-maker should not rely on the daily profit in making decisions

15. Conclusion
The spot market bidding decision–making problem is formulated by a Markov Decision Process
An algorithm is developed to calculate the transition probabilities and reward function
Value iteration method is applied to solve the MDP
Implementation and case study
Three GenCos, GenCo A is the Decision Maker
5 Generators among the 3 GenCos
Description of 2 Examples:
Production Limit without Market Power
Market Power without Production Limit

16. Extension
Risk-neutral decision–maker is assumed. In order to incorporate risk attitude, the variance of decision– maker’s profits should be brought in the bidding decisions. A slightly different formulation, risk– sensitive MDP, can be used to solve this problem.
The objective function is to maximize the total expected future reward during 7-day planning horizon. The 7-day planning horizon is chosen based on the accuracy of the short–term load forecasting. However, an optimal planning horizon should be discussed in the future.
The competitors’ strategies are modeled as price, quantity and probability from the decision–maker ‘s point of view. However, game theory should be incorporated to model how competitors’ activities are affected by decision–maker’s decisions as well.

17. Questions???