1. Smart Grid Big Data: Automating Analysis of Distribution Systems Steve Pascoe steve.pascoe@nisc.coop Manager Business Development E&O - NISC

2. Big Data Analytics – What is it? Big data is a term for data sets that are so large or complex that traditional data processing applications are inadequate. – Challenges include: Analysis, capture, search, sharing, storage, transfer, visualization, querying and security – Typically refers to extracting value not size of the data set. – Tall and wide Migration from Excel to Predictive Analysis

3. Big Data Utilization Evolution

4. Big Data - Distribution Analytics Systems – AMI – SCADA/MV-90 – GIS - OMS Data – Weather – Distribution Models – Interval Data Infrastructure – Cloud v. On-Premise

5. Distribution Analytics for Who? Daily Analysis – Network Management – Near-term Planning Real-time Analysis – Operations Do they want the data or the results?

6. Integrated System Model (ISM) Manage and Analyze the Entire Model As-Designed ISM – “Normal” Network Configuration As-Is ISM – “Current” Model Configuration

7. ISM Model Maintenance Model Requirements – Connectivity – Engineering Parameters ISM QA Apply As-Is to As-Designed Measurements and Forecasts

8. ISM Analysis Automated Daily – Equipment Loading Transformers Devices – Feeder Performance On-Request Real-Time – Fault Location – Suggested Switching

9. Sample Report

11. Time Series Analysis Approach Capture peaks and repeat “offenders” Changes the traditional “single point in time, peak” analysis approach View and analyze interval based results Leverage AMI and primary metering (SCADA/MV90) Industry leading applications, infrastructure and analysis to process 8760 results across the entire ISM network Proactively conduct system-wide time series power flow and in near real-time based on data availability

12. Big Data Solutions Public Cloud – Infrastructure is shared with unknown entities – Not industry specific – Who are your partners? Private Cloud – Industry specific – Strict security controls – Known set of partners

13. Modern Tools for Modern Challenges Relational databases are a versatile, jack of all trades. – Transactional awareness – Great for payment systems, NASDAQ etc, where consistency is critical Big Data data stores tend to be good at specific use cases – Time Series data – Meter measurements, satellite trajectories – Key Value stores – large maps of semi-structured data

14. Database Management Apache Cassandra open source distributed database management – Facebook, Amazon and Google ties Traditional Database Management Solutions – 15,000 to 20,000 transaction a second – Vertical scaling limits exist – Software price scales exponentially (i.e. cores, I/O) Cassandra – 1,000,000 transactions a second – Multiple data streams – Disk space and IO nodes added on the fly – Data is replicated 3-5 times in a data center – Software upgrades have zero downtime,

15. Saclable Data Processing Hadoop – Open sourced by Yahoo – Breaks up large scale analytics on the fly into smaller pieces, analyzes small pieces in parallel. – 1 billion records to analyze and 100 nodes, Each node can analyze 10 million a piece Nearly 100x speed increase Minutes instead of hours Huge ecosystem of tools, ETL and BI Packages

16. The Future of Data Analytics Interval width is shrinking and more channels – Not just our industry More sensors are being deployed Data should come to you, not the other way around – Look for anti-patterns, alerts, unseen-before conditions Mobile enabled Select the Proper Technology

17. Conclusions Cloud computing involving highly detailed, complete network models and associated time series measurements fundamentally changes how utilities perform short term planning and near real-time operations. Management of the As-Designed ISM and the As-Is ISM is critical Automated analysis of this integrated model can help utilities solve existing problem in new ways including; – Enables the ability to capture peaks and repeat “offenders” of circuits and transformers – Helps system operators make near real-time decisions on network configurations to support restoration efforts – Identifies outage sources and fault locations – View and analyze impact of distributed energy resources – Leverage AMI and primary metering (SCADA/MV90) to ensure accurate and reliable network models. In the engineering analysis architecture presented, proactive, smart grid, big data analysis is achieved by performing system-wide, measurement driven, time series power flows in near real-time. This analysis supports management decisions, operations planning, and near real- time operations.

18. Thank You Steve Pascoe steve.pascoe@nisc.coop