IBM T. J. Watson Research Center © 2010 IBM Corporation A Statistical Model for Risk Management of Electric Outage Forecasts Hongfei Li IBM Workshop, September 17, 2010

IBM T. J. Watson Research Center © 2010 IBM Corporation 2 Outline  Overview of statistical applications in Smarter Planet  Project of damage forecasting

IBM T. J. Watson Research Center © 2010 IBM Corporation 3 Overview of Statistical Applications in Smarter Planet  The world is becoming more instrumented and interconnected.  A huge amount of information is created through much denser sensor measurements extending over large space and tracking over time.  Existing methodologies are not sufficient to meet the demand coming from complex problems involving time and space.  Particular spatio-temporal statistical models need to be developed to solve the challenging areas which also follows current IBM business direction as well as professional research direction.

IBM T. J. Watson Research Center © 2010 IBM Corporation 4 Analytics Driven Asset Management (ADAM) Cities are a key focus area in the IBM Smarter Planet Strategy Traffic & Transportation Water availability & purity Building & Energy Safety

IBM T. J. Watson Research Center © 2010 IBM Corporation 5 Key Challenges in Water Management  Water usage has increased at twice the rate of population grow.  Threefold issues: quantity, quality and energy  Interesting Questions: –How can we better schedule our crews to reduce “windshield time”? –How to prevent pipes from breaking? –How can we effectively use capital $ to replace the right bits of our infrastructure? –How can we understand water usage patterns and manage demand? –How can prevent pollution during storm events?

IBM T. J. Watson Research Center © 2010 IBM Corporation 6 Statistical Analysis in ADAM Water usage analysis Pressure zone analysis Failure prediction analysis

IBM T. J. Watson Research Center © 2010 IBM Corporation 7 Green Exploratory Research Spatial-temporal causal modeling for climate change attribution  Motivation: the 2003 European heat wave was one of the hottest summers on record in Europe. The heat wave led to health crises in several countries and combined with drought to create a crop shortfall in Southern Europe. Approximately 35,000 people died as a result of the heat wave.  Goal: address the attribution of extreme climate events, such as heat waves.  Method: develop a novel method to infer causality from spatial-temporal data, as well as a procedure to incorporate extreme value modeling into our method.

IBM T. J. Watson Research Center © 2010 IBM Corporation 8 Environmental Risk Reference: Lozano, A., Li, H., Niculescu-Mizil, A., Liu, Y., Perlich C., Hosking, J. and Abe, N., “Spatial- temporal Causal Modeling for Climate Change Attribution”, Knowledge Discovery and Data Mining conference 2009 Incorporate extreme value modeling into the causality network analysis in order to address the attribution of extreme climate events, such as heatwaves. Figure: Attributing the change in 100-year return level for temperature extremes. Edge thickness represents the causality strength. Figure: Difference of averaged return level between 1967 and 1980 and between 1981 and Investigate return levels changing over time.

IBM T. J. Watson Research Center © 2010 IBM Corporation 9 A Statistical Model for Risk Management of Electric Outage Forecasts Goals Investigate and forecast severe storm impacts on electric infrastructure distribution system

IBM T. J. Watson Research Center © 2010 IBM Corporation 10 A Statistical Model for Risk Management of Electric Outage Forecasts High resolution numerical weather prediction, advanced data assimilation and visualization with applications for severe storms affecting urban areas Quantification of forecast uncertainty caused by various data sources and different modeling structures Uncertainty visualization for operational decision making Weather prediction Damage prediction Restoration time prediction Resource requirement prediction

IBM T. J. Watson Research Center © 2010 IBM Corporation 11 Challenges  Damage forecast model inputs –Which weather inputs are important for damage forecast? –Most weather variables are correlated –Multicollinearity may cause invalid interpretation of weather predictors  Weather forecast calibration –Forecasted variables (e.g., wind speed) may differ in meaning vs. observations used in the damage-forecast-model training –How should physical model outputs be calibrated so that they can be used as the inputs of damage forecast model?

IBM T. J. Watson Research Center © 2010 IBM Corporation 12 Challenges (cont’d)  Gust speed calculation –Exploratory data analysis indicates that gust speed has a stronger relationship to damages vs. wind speed –However, general meteorological models do not provide a direct gust forecast –How should gust speed be calculated based on limited weather information?  Uncertainty quantification and visualization –Uncertainties come from various data sources and different model structures

IBM T. J. Watson Research Center © 2010 IBM Corporation 13 Challenges  Model integration –How should damage forecasts, multiple spatial resolution interpolations and calibration be integrated in one framework? Utility Service Area, AWS/WeatherBug Stations and Example NWP Grid

IBM T. J. Watson Research Center © 2010 IBM Corporation 14 Approach  A damage forecast model at the area substation level is developed using historical weather observations* and outage data** by building a hierarchical Poisson regression model  This damage forecast model is coupled to the meso-g-scale numerical forecasts generated by the “Deep Thunder” (DT) system developed at the IBM Thomas J. Watson Research Center  DT “gust calculation” is developed via a statistical model using time series analysis based on historical DT wind forecast and gust “observations”  Statistical hierarchical modeling integrates various data sources in one model and allows variances or uncertainties analyzed at different levels Historical Damage Data DT Damage Forecast Model Model Training Calibrated DT RAMS/WRF Outputs Historical Weather Data DT Damage Forecast Outputs Data Flow for the Deep Thunder Damage Forecast Model

IBM T. J. Watson Research Center © 2010 IBM Corporation 15 Exploratory Analysis Gust speed has a stronger correlation with damage vs. wind speed Gust speed is adjusted by leaf coverage and ground saturation Number of outages for each substation area is adjusted by infrastructure density

IBM T. J. Watson Research Center © 2010 IBM Corporation 16 Exploratory Analysis Hourly damage/power outage distributions for three major storm events The blue curves show hourly gust speeds These figures provide tools for damage forecasts of sequential days

IBM T. J. Watson Research Center © 2010 IBM Corporation 17 Statistical Hierarchical Modeling is the baseline number of outages, and reflects the infrastructure density in the substation area.

IBM T. J. Watson Research Center © 2010 IBM Corporation 18 Statistical Hierarchical Modeling II

IBM T. J. Watson Research Center © 2010 IBM Corporation 19 Statistical Hierarchical Modeling III Procedure II: Hierarchical Model for Gust Calculation

IBM T. J. Watson Research Center © 2010 IBM Corporation 20 Results Site-Specific DT Gust Forecast Damage Forecast vs. Actual Damage

IBM T. J. Watson Research Center © 2010 IBM Corporation 21 Example Case Study – Retrospective Analysis  02 September 2006 Extra-tropical Event  Remnants of Tropical Storm Ernesto brought heavy rain and gusty winds of 40 to 57 mph across Long Island and southeastern New York, east of the Hudson River, including most of New York City  Numerous trees and power lines down with many power outages reported  There was widespread disruption of transportation systems (e.g., road closures, flooded subways, airport delays) and significant flooding in several regions Reported Rainfall Westchester 0.5" - 2" Kings 0.5" - 1" Queens 0.5" - 1" Richmond 0.5" - 1" Nassau 0.5" " Suffolk 0.5" " Reported Wind Gust Maximums (MPH) Westchester 48Kings 52 Queens 46-51Richmond 49 Nassau 41-57Suffolk 40-55

IBM T. J. Watson Research Center © 2010 IBM Corporation 22 DT Weather Forecast Rainfall Forecast Wind Forecast

IBM T. J. Watson Research Center © 2010 IBM Corporation 23 Damage Forecasting Result Actual OutagesOutage EstimateOutage Upper Bound

IBM T. J. Watson Research Center © 2010 IBM Corporation 24 Probability of Outages per Substation Area for Various Ranges of Outages (Texturing of Outage Color Illustrates Probability with Value)

IBM T. J. Watson Research Center © 2010 IBM Corporation 25 References:  Li, H., Treinish, L. and Hosking, J., “A Statistical Model for Risk Management of Electric Outage Forecasts”, IBM Journal of Research and Development, vol. 54, no. 3, 2010