Testing Strategic Models of Firm Behavior in Restructured Electricity Markets: A Case Study of ERCOT Ali Hortacsu, University of Chicago Steve Puller, Texas A&M

Motivation Empirical auction literature –Bid data + equilibrium model  valuation “New Empirical IO” –Eqbm (p,q) data + demand elasticity + behavioral assumption  MC Can equilibrium models be tested? –Laboratory experiments Electricity markets are a great place to study firm pricing behavior This paper measures deviations from theoretical benchmark & explores reasons

Texas Electricity Market Largest electric grid control area in U.S. (ERCOT) Market opened August 2001 Incumbents –Implicit contracts to serve non-switching customers at regulated price Various merchant generators

Electricity Market Mechanics Forward contracting –Generators contract w/ buyers beforehand for a delivery quantity and price –Day before production: fixed quantities of supply and demand are scheduled w/ grid operator –(Generators may be net short or long on their contract quantity) Spot (balancing) market –Centralized market to balance realized demand with scheduled supply –Generators submit “supply functions” to increase or decrease production from day-ahead schedule

Balancing Energy Market Spot market run in “real-time” to balance supply (generation) and demand (load) –Adjusts for demand and cost shocks (e.g. weather, plant outage) Approx 2-5% of energy traded (“up” and “down”) –“up”  bidding price to receive to produce more –“down”  bidding price to pay to produce less Uniform-price auction using hourly portfolio bids that clear every 15-minute interval Bids: monotonic step functions with up to 40 “elbow points” (20 up and 20 down) Market separated into zones if transmission lines congested – we focus on uncongested hours

Who are the Players? Generator% of Installed Capacity TXU Electric24 Reliant Energy18 City of San Antonio Public Service8 Central Power & Light7 City of Austin6 Calpine5 Lower Colorado River Authority4 Lamar Power Partners4 Guadalupe Power Partners2 West Texas Utilities2 Midlothian Energy2 Dow Chemical1 Brazos Electric Power Coop1 Others16

Incentives to Exercise Market Power Suppose no further contract obligations upon entering balancing market INCremental demand periods –Bid above MC to raise revenue on inframarginal sales –Just “monopolist on residual demand” DECremental demand periods –Bid below MC to reduce output –Make yourself “short” but drive down the price of buying your short position (monopsony)

Price Quantity RD i (p) S i (p) MC i (q) MR i (p) QC i A B C E D

Methods to Test Expected Profit Maximizing Behavior Difficult to compare actual to ex-ante optimal bids –Wolak (2000,2001)  solving ex-ante optimal bid strategy (under equilibrium beliefs about uncertainty) is computationally difficult Options 1)Restrict economic environment so ex-post optimal = ex-ante optimal Intuitively, uncertainty and private information shift RD in parallel fashion 2)Check (local) optimality of observed bids (Wolak, 2001) Do bids violate F.O.C. of E ε [π(p,ε)] ? 3)Can simple trading rules improve upon realized profits?

Overview of Model Setup –Static game, N firms –Marginal Cost i is public information –Contract quantity (QC i ) and price (PC i ) are private information –Generators bid supply functions S i (p,QC i ) Sources of uncertainty –Total demand D(p) stochastic –Rivals’ bids S -i (QC -i )  Market clearing price (p c ) is uncertain (application of Wilson 1979 share auction)

Sample Genscape Interface

Overview of Model (contd)

Computing Ex Post Optimal Bids Ex post best response is Bayesian Nash Eqbm  Uncertainty shifts residual demand parallel in & out  Can trace out ex post optimal / equilibrium bids

Data (Sept 2001 thru July 2002) Bids –Hourly firm-level bids Demand in balancing market – assumed perfectly inelastic Marginal Costs for each operating fossil fuel unit Fuel efficiency – average “heat rates” Fuel costs – daily natural gas spot prices & monthly average coal spot prices Variable O&M SO2 permit costs –Each unit’s daily capacity & day-ahead schedule

Measuring Marginal Cost in Balancing Market Use coal and gas-fired generating units that are “on” and the daily capacity declaration Calculate how much generation from those units is already scheduled == Day-Ahead Schedule Total MC Residual MC Day-Ahead Schedule Price MW

Reliant (biggest seller) Example

TXU (2 nd biggest seller) Example

Guadalupe (small seller) Example

Calculating Deviation from Optimal Producer Surplus Optimal Actual Avoid $

Measures of Foregone Profits

Percent of Potential Gains from Not Bidding

Learning by Larger Players?

Testing Expected Profit Maximizing Behavior 1)Restrict economic environment so ex-post optimal = ex-ante optimal 2)No restrictions -- Check (local) optimality of observed bids (Wolak, 2001) 3)Can simple trading rules improve upon realized profits?

Generator’s Ex-Ante Problem Max E ε [π(p,ε)] s.t. (1) monotonicity of bids (2) transmission congestion (3) physical operating contraints Restrict our sample  ignore constraints Wolak test for (local) optimality: –H o : Each bidpoint chosen optimally –Changing the price of each (p k,q k ) will not incrementally increase profits

Reliant (biggest seller) Example

Guadalupe (small seller) Example

Test for (Local) Optimality of Bids Moments for GMM:

Test for (Local) Optimality of Bids Fail to reject (even for Guadalupe!) Test is lower power in our setting –Future work: use *quantity* moments FirmJ-statd.o.f.p-value Reliant TXU Guadalupe

“Naïve Best Reply Test” of Optimality Bidders can see aggregate bids with a few day lag Simple trading rule: use bid data from t-3, assume rivals don’t change bids, and find ex post optimal bids (under parallel shift assumption) Does this outperform actual bidding?

How Much Does Trading Rule Increase Profits? Bryan$200/hr18 Calpine$1,325/hr18 City of Austin$1,129/hr18 Reliant$957/hr18 TXU$1,770/hr18 Preliminary

Learning in Second Year? Sep01-Jul02 Sep02- May03 Reliant/Texas Genco82%27% TXU53%77% Bryan44%56% Calpine34%44% City of Austin27%28% Note: Second year excludes Aug’02, Dec’02 (data not clean yet) and Feb ’03 (“weather incident”) Fair bit of month to month variability by firm.

What the Traders Say about Suboptimal Bidding 1.Lack of sophistication at beginning of market Some firms’ bidders have no trading experience; are employees brought over from generation & distribution 2.Heuristics Most don’t think in terms of “residual demand” Rival supply not entirely transparent b/c Eqbm mapping of rival costs to bids too sophisticated Some firms do not use lagged aggregate bid data Bid in a markup & have guess where price will be 3.Newer generators If a unit has debt to pay off, bidders follow a formula of % markup to add

What the Traders Say (contd) 4.TXU “old school” – would prefer to serve it’s customers with own expensive generation rather than buy cheaper power from market Anecdotal evidence that relying more on market in 2 nd year of market 5.Small players (e.g. munis) “scared of market” – afraid of being short w/ high prices Don’t want to bid extra capacity into market because they want extra capacity available in case a unit goes down

Counterfactual Welfare Calculations (Not Yet Completed) Productive inefficiencies under alternative bidding (1) Actual vs. Competitive (Vickrey multiunit) (2) Actual vs. Unilateral Best-Reply (Uniform-Price) (3) Actual vs. "Large Unilateral" and "Small Competitive"

Conclusion Electricity markets are a great “field” setting to understand firm behavior under uncertainty and private information Stakes appear to matter in strategic sophistication Both sophistication (“market power”) and lack of sophistication (“avoid the market”) contribute to inefficiency in this market

The End

Dispersion of “Money on the Table” Reliant

Dispersion of “Money on the Table” ReliantTXU CalpineBryan

Quantity Traded in Balancing Market Mean = -257 Stdev = 1035 Min = th Pctile = th Pctile = 390 Max = 2713 Sample: Sept 2001-July 2002, 6:00-6:15pm, weekdays, no transmission congestion

Zones in ERCOT 2002 Source: Public Utility Commission of Texas, MOD Annual Report (2003)

Sample Bidding Interface

Do We Expect to See Optimal Bidding? First year of market –Some traders experienced while others brought over from generation and transmission sectors Many bidding & optimization decisions being made Real-time information? –Frequency charts & Genscape sensor data  rival costs –Aggregate bid stacks with 2-3 day lag  “adaptive best-response” bidding? Is there enough $$ at stake in balancing market? –Several hundred to several thousand per hour “Bounded rationality”

Smaller Players Appear to bid to “withhold capacity” to avoid the balancing market  productive inefficiencies Not unilateral market power because markups/markdowns are too large given their small inframarginal sales Policy implications: –Fixed costs to participation? –But some small players are closer to optimal Bidders lacking trading experience? Sticky market for managerial efficiency?

“Testing” Explanations for Suboptimal Bidding 1.Not enough $$ at stake  avoid the balancing market –Potential profits for each 6-7pm Reliant = $6,165 Lamar Power Partners = $1,391 But Bryan = $315!! 2.Learning –Exercising market power on DEC side (“monopsony”) may not be obvious Bid to DEC low so you’re short but at a low price –Decrease in bid-ask spread –Profitability over time –Use more bid points over time

“Testing” Explanations (contd) 3.Adjustment costs? Marginal generating unit most often is gas (very flexible) 4.Transmission congestion is important –We analyze only periods with no interzonal transmission congestion –Congestion changes residual demand –If cannot perfectly forecast congestion, the bidding strategy under congestion may “spillover” to uncongested times 5.Collusion? Would be small(!) players - unlikely

Sample Bidder’s Operations Interface

Residual Contract Positions

Difference in average system loads: INC = 33GW DEC=29GW Can marginal costs differ by that much?

Medians of Reduced-form ‘conduct’ measures

Example of Data We See Sept 14, :00-6:15pm Total Balancing Demand = -996 MW Aggregate Bids and MC One Firm’s Bids and MC

Calpine (3 rd biggest seller) Example

Test for (Local) Optimality of Bids

Who are the Players? GeneratorAverage Balancing Sales** (MWh) % of Installed Capacity TXU Electric15624 Reliant Energy47318 City of San Antonio Public Service*8 Central Power & Light287 City of Austin406 Calpine785 Lower Colorado River Authority*4 Lamar Power Partners234 Guadalupe Power Partners82 West Texas Utilities102 Midlothian Energy*2 Dow Chemical*1 Brazos Electric Power Coop51 Others*16 * Cannot uniquely identify the bids ** Sales in zones where bids can be uniquely identified