by Zinour Charifoulline, Scott Rowan Automation of Hardware Commissioning - Offline Analysis of 60A circuits

Test Procedures/Offline Analysis Automated Analysis Walkthrough Automated Analysis Results PCC.1 PNO.d1 Inconsistencies EPC Feedback Summary/What Next? Outline S. Rowan - MPE-TM - 27/02/2013

Test Procedures/Offline Analysis S. Rowan - MPE-TM - 27/02/2013

Tests Procedures exist for all circuit types by current level Defines all powering cycles to fully commission circuits Defines offline analysis criteria to be checked and signed by experts Pass Criteria is seemingly simple enough to automate Test Procedures S. Rowan - MPE-TM - 27/02/2013

Using the eDSL Java Interface pass criteria can be written in an easy to read/understand sentence form Offline Analysis ±PCC.1 //Verify converter current to be +3.7A/+-0.6A 60seconds after the 1st crowbar activation (2 nd PM) assertThat(I_MEAS).isEqualTo(+3.7, AMPERE).withinAbs(0.6, AMPERE).at(60, SECOND).after(Occurrence.SECOND, PM_EVENT_TRIGGER); //Verify converter current & voltage decay is an exponential function with an absolute tolerance of 0.1A/0.05V starting 20ms after the 1st crowbar activation (2 nd PM) assertThat(I_MEAS).isEqualTo(EXP).withinAbs(0.1, AMPERE).starting(20, MILLI(SECOND)).after(Occurrence.SECOND, PM_EVENT_TRIGGER).ending(150, SECOND).after(Occurrence.SECOND, PM_EVENT_TRIGGER); assertThat(V_MEAS).isEqualTo(EXP).withinAbs(0.1, VOLT)… … S. Rowan - MPE-TM - 27/02/2013

Using the eDSL Java Interface pass criteria can be written in an easy to read/understand sentence form Offline Analysis ±PCC.1 //Verify 1st PM-trigger exist and offset is zero assertThat(I_MEAS).isEqualTo(0.0, AMPERE).withinAbs(0.1, AMPERE).at(0, MILLI(SECOND)).after(Occurrence.FIRST, PM_EVENT_TRIGGER); //Verify start point at I_PCC=10A assertThat(I_MEAS).isEqualTo(10.0, AMPERE).withinRel(1.0, PERCENT).at(1, SECOND).before(Occurrence.SECOND, PM_EVENT_TRIGGER); Several additional checks are added to improve robustness S. Rowan - MPE-TM - 27/02/2013 Question! Can this be used by a non-java expert??

Automated Analysis Walkthrough S. Rowan - MPE-TM - 27/02/2013

PCC.1 Automated Analysis Walkthrough Offline Analysis ±PCC.1 Verify 1st PM-trigger exists and offset is zero S. Rowan - MPE-TM - 27/02/2013

PCC.1 Automated Analysis Walkthrough Offline Analysis ±PCC.1 Verify plateau current I_PCC=10A S. Rowan - MPE-TM - 27/02/2013

PCC.1 Automated Analysis Walkthrough Offline Analysis ±PCC.1 Verify converter current to be +3.7A/+-0.6A 60sec after the 1st crowbar activation S. Rowan - TE-MPE-PE - 27/12/2013

PCC.1 Automated Analysis Walkthrough Offline Analysis ±PCC.1 Verify current decay is an exponential function with an absolute tolerance of 0.1A after the 1st crowbar activation S. Rowan - MPE-TM - 27/02/2013

PCC.1 Automated Analysis Walkthrough Offline Analysis ±PCC.1 Verify voltage decay is an exponential function with an absolute tolerance of 0.05V after the 1st crowbar activation Then simply repeat for tests for negative cycle!!! S. Rowan - MPE-TM - 27/02/2013

Automated Analysis Results S. Rowan - MPE-TM - 27/02/2013

PCC.1 Automated Analysis Results 2011 Automated Analysis Results ±PCC HWC campaign 233 PCC.1 tests on A-circuits 17 consistency fails 10 failed due to I_ERR_PCC which is not in offline analysis 3 cases due “just” failing decay criteria 92.7% of consistency HWC campaign 233 PCC.1 tests on A-circuits 17 consistency fails 10 failed due to I_ERR_PCC which is not in offline analysis 3 cases due “just” failing decay criteria 92.7% of consistency. S. Rowan - MPE-TM - 27/02/2013

PCC.1 Automated Analysis Results Automated Analysis Results ±PCC HWC campaign 398 PCC.1 tests on A-circuits 10 consistency fails 1 was “successful” but “tuning to be done”? 8 cases due to “just” failing decay criteria 1 anomaly accepted manually??? 97.5% of consistency! HWC campaign 398 PCC.1 tests on A-circuits 10 consistency fails 1 was “successful” but “tuning to be done”? 8 cases due to “just” failing decay criteria 1 anomaly accepted manually??? 97.5% of consistency! Should have failed! Circuit: RCBV19.L5B1 Should have failed! Circuit: RCBV19.L5B1 S. Rowan - MPE-TM - 27/02/2013

2011 HWC campaign 1022 PNO.d1 tests on A-circuits 69 consistency fails 58 cases due to “just” failing decay criteria 8 of which due to crowbar voltage 2 current plateau checks failed 93.2% of consistency! 2011 HWC campaign 1022 PNO.d1 tests on A-circuits 69 consistency fails 58 cases due to “just” failing decay criteria 8 of which due to crowbar voltage 2 current plateau checks failed 93.2% of consistency! PNO.d1 Automated Analysis Results 2011 Automated Analysis Results ±PNO.d1 S. Rowan - MPE-TM - 27/02/2013

PNO.d1 Automated Analysis Results Automated Analysis Results ±PNO.d HWC campaign 997 PNO.d1 tests on A-circuits 97 consistency fails 64 cases due to “just” failing decay criteria 20 due to failing crowbar voltage check 1 significantly bad circuit (same as PCC.1) 12 based on data errors (passed manually) 90.3% of consistency! HWC campaign 997 PNO.d1 tests on A-circuits 97 consistency fails 64 cases due to “just” failing decay criteria 20 due to failing crowbar voltage check 1 significantly bad circuit (same as PCC.1) 12 based on data errors (passed manually) 90.3% of consistency! S. Rowan - MPE-TM - 27/02/2013 Should have failed! Circuit: RCBV19.L5B1 Should have failed! Circuit: RCBV19.L5B1

Inconsistencies S. Rowan - MPE-TM - 27/02/2013

~60% of inconsistencies (4% of all tests) due to high circuit resistance, resulting in decay being outwith the defined criteria Inconsistencies - Current decay All should pass the criteria 27 circuits should fail with current criteria R_TOTAL>0.0675Ω 27 circuits should fail with current criteria R_TOTAL>0.0675Ω Possible solutions: 1.Refine thresholds: 3.7±0.6A => 3.4±0.5A 2.Use Layout DB to compare decay constant S. Rowan - MPE-TM - 27/02/2013 Layout Database values

~30% of inconsistencies (2% of all tests) due to voltage limits across crowbar Most failures due to strict criteria and/or noise in signal. Inconsistencies – Crowbar Voltage limits Possible solutions: 1.Relax thresholds: 1.1±0.1A => 1.1±0.2A 2.Check average of plateau lies within limits S. Rowan - MPE-TM - 27/02/2013

EPC Feedback S. Rowan - MPE-TM – 27/02/2013

Consistency checks «correspondent vraiment» to what is expected Consistency check over previous campaigns resulted in discovering a faulty module (RCBV19.L5B1) that EPC were unaware of! Regarding pass criteria - EPC accept all proposed solutions With existing pass criteria the automated analysis has >90% consistency With relaxed pass criteria 60 A circuit analysis has >99% consistency Regarding possibly more robust assertions Comparison to a modelled reference curve? – difficult to implement Plateau averages? – possibility, though accuracy/necessity questionable Time constant check on decay? – to be discussed, will require Layout DB Ability to use parameters from Layout Database? – “soon™” EPC Feedback S. Rowan - MPE-TM - 27/02/2013

A simple qualification check was requested for 3 known faults. Note: All had already been flagged by consistency check. EPC Feedback S. Rowan - MPE-TM - 27/02/2013 RCBH13.L6B1:PNO.d1 Fails, even with relaxed criteria

A simple qualification check was requested for 3 known faults. Note: All had already been flagged by consistency check. EPC Feedback S. Rowan - MPE-TM - 27/02/2013 Fails, even with relaxed criteria RCBV30.L4B1:PCC.1

RCBV24.L4B1:PNO.d1 A simple qualification check was requested for 3 known faults. Note: All had already been flagged by consistency check. EPC Feedback S. Rowan - MPE-TM - 27/02/2013 Interesting curve Passes 25A check point but fails 8A Must be taken into account when defining new criteria

Summary/What Next? S. Rowan - MPE-TM - 27/02/2013

Framework and language clearly usable by non-java experts! 60 A circuit offline analysis can now be fully automated (PCC1, PNO.d1) Will be ready for upcoming HWC campaign With relaxed pass criteria 60 A circuit analysis has >99% consistency Pass criteria to be finalised/documented in test procedures All test procedures currently being updated by MP3 Due to the discovery of an unknown fault, we propose to continue and carry out consistency checks for all remaining numerous circuit types ( A & 600 A). May find further unknown system faults Will assist in refining pass criteria, useful for manual analysis Analysis not likely to be automated during coming HWC, but if the algorithms are written and qualified, future implementation is a possibility. Summary S. Rowan - TE-MPE-PE - 11/12/2013

Thanks for listening! S. Rowan - MPE-TM - 27/02/2013