1. June 27, 2008 NDSWG ERCOT and TSP activities discussion for SEM Go-Live

2. 2 Agenda Review SEM Go-Live process and steps involved ERCOT model validation –CIM Audit and validation rules –Data consistency check ERCOT model validation timeline TSP data validation activities discussion Introduction to CIMTool

3. 3 SEM Go-Live process – Steps involved # 1ERCOT freezes CIM schema 2ERCOT validates model 3SEM Go-Live - NMMS and ERCOT validated model deployed to Production environment 4TSPs perform data validation 5First Notice to Market Participants for enabling Nodal Protocol § 3 6Second Notice to Market Participants for enabling Nodal Protocol § 3 7ERCOT enables Nodal Protocol § 3

4. 4 Agenda Review SEM Go-Live process and steps involved ERCOT model validation –CIM Audit and Validation rules –Data consistency check ERCOT model validation timeline TSP data validation activities discussion Introduction to CIMTool

5. 5 ERCOT CIM Audit 1.Inventory all modules in downstream systems that use CIM models –MMS Reviewing data and resolving data issues with MMS team Identifying all fields in MMS profile with no data –Identifying source of data –If data not currently available, creating temporary data to provide quick turnaround for the vendor to work with –EMS Started similar process for EMS AREVA reporting initial data elements needs Anticipating number of reported issues to ramp up as they continue working on their EMS CIM importer –Inventory remaining users of CIM model – Settlements and Billing, Outage Scheduler 2.Identify classes, associations, and attributes in existing CIM model needed by downstream systems 3.Validate needed classes, associations and attributes are in defined NMMS profiles 4.Identify missing data required and complete model change request Objective of CIM Audit: Ensure that all model requirements of Nodal systems (MMS, EMS, OS etc.) is incorporated in the CIM model by reviewing application databases.

6. 6 Rules based validation ERCOT is/will perform the following tasks prior to SEM Go-Live 1.Gather requirements for validation rules 2.Design validation rule capability -manual (i.e., flowgates, DC ties) -bulk data validation (i.e., ownership, limits, hubs) -NMMS NOMCR/SAMR validation (i.e., ownership, limits) 3. Execute bulk data validation 4. Test NMMS NOMCR/SAMR validation

7. 7 Agenda Review SEM Go-Live process and steps involved ERCOT model validation –CIM Audit and validation rules –Data consistency check ERCOT model validation timeline TSP data validation activities discussion Introduction to CIMTool

8. 8 ERCOT shall perform data consistency check prior to SEM Go-Live The objective is to jump start the nodal model building process by leveraging as much of the zonal model as possible to seed NMMS. It will also ensure consistency of model data and provide for comparison of application results between Zonal and Nodal EMS systems First step: Import zonal model into NMMS Second Step: Database comparison: Three-way database comparison –Zonal Savecases –Nodal Savecases created through conversion scripts on Zonal Savecases –Nodal Savecases generated through EMS CIM importer Result: Ensures Zonal and Nodal data are similar and validates that all data is populated correctly Third Step: Application comparison: Execute Nodal EMS system on EMS CIM importer generated Savecases and compare results with Zonal EMS system results Result: Ensures that Nodal EMS system executes based on Nodal CIM model and produces results that are equal to (or better than) Zonal EMS system Upon completion of import and validation steps, the model shall be deemed ready for Single Entry Model Go-Live NMMS Zonal EMS SCADASE MTLFGEN Seed Model + Nodal data PFCA

9. 9 Data consistency check – Background NMMS Nodal EMS CIM Importer Seed Model (CIM XML) SCADANET OAGCTGS GENRAS DYNRTG Zonal EMS SCADANET OAG*CTGS GENRAS** DYNRTG More Nodal data (Registration, MMS, NMMS, etc.) * - OAGMOM is drastically different between Zonal and Nodal, as of now, only Zonal OAGMOM is exported ** - RASMOM has no Zonal equivalence, a Nodal RASMOM was hand built using Zonal SPS data Conversion Scripts Nodal Test Data EDS Nodal EMS (interim test data) SCADANET GENDYNRTG ALRM Genesys Archive End Goal CIM XML

10. 10 Data consistency check – Data Comparison The following two database comparisons will be performed Comparison 1: Zonal Savecases data compared with Nodal Savecases created through EMS CIM importer Comparison 2: Nodal Savecases created by running conversion scripts on Zonal data, compared with Nodal Savecases created through EMS CIM importer DatabaseComparisonMethod OAGMOMZonal with CIM Importer outputCommon attributes only Zonal converted to Nodal with CIM importer outputHabitat tool SCADAMOMZonal with CIM Importer outputCommon attributes only Zonal converted to Nodal with CIM importer outputHabitat tool GENMOMZonal with CIM Importer outputCommon attributes only Zonal converted to Nodal with CIM importer outputHabitat tool NETMOMZonal with CIM Importer outputCommon attributes only Zonal converted to Nodal with CIM importer outputHabitat tool DYNRTGZonal to CIM importer outputData exported from Application CTGSZonal to CIM importer outputData exported from Application RASMOMZonal converted to Nodal with CIM importer outputCommon attributes The following methods will be used to compare the cases Method 1: Habitat tools will be used to compare Habitat Savecases if the two cases are not drastically different Method 2: Common attributes between Zonal and Nodal Savecases will be compare manually Method 3: Application results will be exported to text files and compared manually Different databases, and often even different tables within the same database, will call for different strategies

11. 11 Data consistency check – Applications NMMS Zonal EMSNodal EMS Compare results SCADASE MTLFGEN CIM Importer Seed Model + Nodal data SCADASE MTLFGSS EMS applications such as Power Flow, SE, Contingency Analysis are run in both Zonal and Nodal EMS systems and results are compared Power Flow: Power Flow cases are compared at various load levels SE: SESTATS is run in both Zonal and Nodal and results are compared over a six hour period. In addition, comparisons between SE estimated bus voltage and angles, branch flows, injections are carried out analyzing data from different times of the day SCADA: Telemetries and calculations are compared between Zonal and Nodal MTLF: Statistics will be developed by comparing data for seven days CA: Contingency analysis with RAS will be carried out in real time or study mode and results will be compared PFCAPFCA

12. 12 Model Build and Validation - Challenges Data comparison challenges Data hierarchy differences Habitat differences because of extra fields in Zonal and Nodal EMS systems Data differences arising from data sourced from other systems (Registration, MMS etc) for Nodal Application validation challenges Data inputs into Nodal and Zonal EMS systems are not fully identical Applications in Nodal may include new features that affect results New applications that have been developed for Nodal cannot be compared with anything The above challenges result in a complex data consistency check process, requiring significant effort and time

13. 13 Agenda Review SEM Go-Live process and steps involved ERCOT model validation –CIM Audit and validation rules –Data consistency check ERCOT model validation timeline TSP data validation activities discussion Introduction to CIMTool

14. 14 Model validation sequence ERCOT will conduct a NMMS performance test prior to SEM Go-Live Rules validation can occur parallel to the Data consistency check effort Once both activities are complete, ERCOT will initiate the approval sequence for SEM Go-Live

15. 15 Agenda Review SEM Go-Live process and steps involved ERCOT model validation –CIM Audit and validation rules –Data consistency check ERCOT model validation timeline TSP data validation activities discussion Introduction to CIMTool and supporting documentation

16. 16 TSP activities after SEM Go-Live Objectives of TSP activities: 1.Meet Nodal Protocol and Transition Plan requirements for TSPs submittal of data to ERCOT for its Transmission Elements –Protocols Section 3.10.7.1: Modeling of Transmission Elements and Parameters (1) ERCOT, each TSP, and each Resource Entity shall coordinate to define each Transmission Element such that the TSP’s control center operational model and ERCOT’s Network Operations Model are consistent. Protocols Section 3.10.7.1.1, 3.10.7.1.2, 3.10.7.1.3, 3.10.7.1.4, 3.10.7.1.5 –Typical language…(2) each TSP shall provide ERCOT with the following information … –Language requiring TSPs to provide for all other remaining “Transmission Elements” only excluding Generation Resources information which comes from Resource Entities –Protocols Section 3.10.7.2 (4), 3.10.7.2 (7) TSPs shall provide ERCOT with information describing all transmission Load connections.. Each TSP shall also provide information to ERCOT describing automatic Load transfer (rollover) plans.. –Protocols Section 3.10.4 (7) A TSP, with ERCOT’s assistance, shall validate its portion of the Network Operations Model according to the timeline provided in Section 3.10.1. ERCOT shall provide TSPs access … –ERCOT Nodal Transition Plan - Section 5.4.1: Data and Telemetry Testing Requirements (2) ERCOT shall develop procedures for setup and data implementation requirements applicable to TSPs and QSEs. ERCOT shall with the assistance of TSPs and QSEs: –(a) Verify that all data and telemetry requirements specified in the nodal Protocols Section 3.10, Network Operations Modeling and Telemetry, have been met and that all data submittals are complete and represent an accurate model of the ERCOT System. … –Other Protocol references: 3.10 (1) & (3), 3.10.5, See appendix Are there any additional Protocol sections that will have to be considered?

17. 17 TSP activities – suggestions from TSPs Step 1: Finalize and freeze TSP model –Finalize code changes for CIM model –Finalize database conversion process from Legacy EMS to CIM –Launch Network Model Management process Step 2: Model synchronization and refinement –Synchronize CIM RDFIDs using ERCOT and TSP RDFID –Synchronize substation and equipment names –Modify ERCOT model to add classes that facilitate on-going model synchronization –Add missing objects in the model. E.g., Series devices, zero-impedance lines –Verify Ownership / Operatorship –Verify issues with Substation aggregation –Verify station id and mnemonics - is station modeled in the correct weather zone, and are GPS coordinates for station correct. ERCOT recommends that the foundation of TSP activities should be driven by Protocol Sections. Following are some of the suggested activities that ERCOT has received from TSPs.

18. 18 TSP activities – suggestions from TSPs contd.. Step 2: Model synchronization and refinement – Contd.. –Verify switches operating code or number - are all switching that could interrupt flow modeled, are all switches that could change station topology modeled? –Verify Circuit Breakers operating code or number - are all CBs that could interrupt flow modeled, are all CBs that could change station topology modeled? –Verify Transformers IDs and models – Taps, LTC, regulation –Verify the connectivity of device internal to the station. –Verify Line names and impedances, ratings and Dynamic ratings –Verify mapping of ICCP data to the model –Verify the characteristics and IDs of other devices that might be modeled internal to a station –Verify the association of node to connectivity groups and the PSS/E bus number association to the group. –Maintain connectivity node group definitions to support PSS/E file exchange –Verify conversion of operation model to PSS/E model Step 3: Verify status of in-flight Service Requests –Verify implementation status of previously submitted Service Requests –Verify implementation in both Zonal and Nodal systems

19. 19 ERCOT will provide TSPs supporting documentation prior to SEM Go-Live CIM data dictionary and schema –Will post V 1.17 of CIM data dictionary on Nodal Readiness Center by COB June 27, 2008 –Expect updates to the data dictionary as schema changes NMMS Modeling Guidelines Document – currently in development –ERCOT internal NOMCR process –New data types –Rules used in creating electrical buses and resource nodes –Minimum CIM and ERCOT data requirements for all device types –ICCP data submittal process NMMS Validation Rules Document – currently in development –Translation of the ERCOT-implemented NOMCR/SAMR validation rules ERCOT will provide an update on document status at the NDSWG on July 15, 2008

20. 20 What can TSPs do immediately? Download CIM data dictionary and schema from Nodal Readiness Center Download ERCOT CIM data from Transmission planning website (http://oldercot.ercot.com/tmaps/login.cfm). CIM data will be available for download by COB July 02, 2008http://oldercot.ercot.com/tmaps/login.cfm Investigate methods and tools to parse CIM data

21. 21 Agenda Review SEM Go-Live process and steps involved ERCOT model validation –CIM Audit and validation rules –Data consistency check ERCOT model validation timeline TSP data validation activities discussion Introduction to CIMTool

22. 22 TSP Validation Tool Recommendation ERCOT has researched and found the CIMTool (freeware at http://cimtool.org) to perform NOMCR validation before submitting to NMMShttp://cimtool.org CIMTool is an Eclipse (www.eclipse.org) plug-in and provides the means to:www.eclipse.org –Validate profiles –Validate instance files against a profile Validation of the model occurs when a model is imported Rules can be edited and extended using an extension of the Jena rule language –Validate incremental files against an instance file and a profile –Define profiles from a UML model –Import profiles from a spreadsheet –Generate XML and RDF schemas from a profile –… and more ERCOT will provide training on the use of CIMTool in July/August timeframe

23. 23 Sample CIM/Tool Screen – Diagnostic Screen

24. 24 Example Validation Errors Undefined classes Undefined properties Instantiation of abstract classes Illegal cardinality on an association (e.g. more or less associations than expected) Range violation for a data item Illegal domain of a property Base voltages for conducting equipment connected at a connectivity node do not agree Isolated nodes Untyped objects Missing or extra terminal Duplicate properties Unexpected loop 24

25. 25 Next Steps for the SEM Go-Live Procedure Activity Reach consensus on TSP activities to be performed after SEM Go-Live Publish Whitepaper detailing CIM data validation methodology TSPs investigate the methods to perform model validation activities after SEM Go-Live and provide an estimated duration to ERCOT Send out revised SEM Go-Live procedure to NDSWG and TPTF for comments Seek ROS’ endorsement on SEM Go-Live Procedure Seek TPTF approval for SEM Go-Live Procedure Seek TAC approval for SEM Go-Live Procedure

26. 26 Questions?

27. 27 Appendix Selected Protocol References: 3.10 Network Operations Modeling and Telemetry (1)ERCOT shall use the physical characteristics, ratings, and operational limits of all Transmission Elements of the ERCOT Transmission Grid and other information from the TSPs to specify limits within which the transmission network is defined in the network models made available to Market Participants on the MIS Secure Area and used to operate the ERCOT Transmission Grid as updated. (3) TSPs shall provide ERCOT with equipment ratings and update the ratings as required by ERCOT. ERCOT shall post all equipment ratings on the MIS Secure Area no later than the day prior to the ratings becoming effective including the identity of the Transmission Element, old rating and new rating, effective date, and a text reason supplied by the appropriate TSP(s) for the rating change. ERCOT may request TSPs to provide detailed information on the methodology, including data for determination of each requested rating. ERCOT may review and comment on the methodology. ERCOT shall post all methodologies on the MIS Secure Area within seven days following a change in methodology. 3.10.5 TSP Responsibilities (1)Each TSP shall design, implement, operate, and maintain their systems to meet the TAC- approved ERCOT Telemetry Criteria under Section 3.10.7.5, for measurements facilitating the observability of the Electrical Buses used for SCED. However, there is no obligation to re-construct or retrofit already existing installations except as shown to be needed in order to achieve TAC-approved observability criteria and SE performance standard. (2)TSPs shall add telemetry at ERCOT’s request to maintain observability and redundancy requirements as specified herein, and under Section 3.10.7.5. ERCOT shall request such additions when a lack of data telemetry has caused, or can be demonstrated to result in, inaccuracies between Real-Time measurements and modeling outcomes that could result in incorrect LMP prices or potential reliability problems

28. 28 Appendix - Bulk Data Validation Starting process of validating CIM data in bulk –Loading CIM file into database –Intend to run queries, reports and scripts to perform additional validation –Validation will be similar to the Level1 validation done once NMMS is live –Will not attempt to do “complex” analysis that is better done by comparing powerflow, state estimator, etc. results

29. 29 Appendix - NMMS NOMCR/SAMR Validation Define validation rules that will be performed against each NOMCR/SAMR submittal Implement validation rules within NMMS –NMMS has “hooks” to call validation logic, –ERCOT needs to use these “hooks” to implement the validation logic