1.  The AccTesting Framework - Overview & Visions Kajetan Fuchsberger TE-MPE TM, 2013-04-25 On behalf of the TE-MPE-MS Software Team: M.Audrain, J.-C. Garnier, A.Gorzawski, A. Moscatelli, I.R.Ramirez, J.Suchowski, P.C. Turcu, M. Zerlauth EDMS 1283942

2. 2 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Content OverviewOngoingPlans & Visions

3. 3 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Content OverviewOngoingPlans & Visions

4. 4 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions From Alvaros’ pages to AccTesting  As from now on: “Alvaros’ pages” deprecated  Can we remove them?

5. 5 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Overview e.g.: LabView, Powering Server … e.g.: HWC Sequencer, LHC Sequencer or Custom Components e.g.: Magnet Circuits, BIC/PIC …

6. 6 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions AccTesting Server  Orchestrates the whole process:  Test Execution  Test Analysis  Exclusively reads/persists data in the database.  Notifies all the GUIs about changes.  Robust Design:  Continuously persists relevant data to be able to recover in case of a crash.  Gracefully handles unexpected behaviour of Execution- and Analysis Components.

7. 7 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Workflow I 1. User “expresses his wish” to execute one (or many) tests.

8. 8 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Workflow II 2. The Tests go to the Execution Basket (Server!)

9. 9 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Workflow III 3. The Scheduler (on the Server) will decide when to start which test(s).

10. 10 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Why so complicated?  Central Scheduling can respect all the conditions, even if requests come from different GUIs.  When conditions are fulfilled later, the tests are started automatically (No delays).  No Need for reservation of circuits anymore.  More dynamic

11. 11 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Designed for Extension  Server Extension Points:  TestStepHandler (handle certain types of SystemTests)  Constraints (Restrict Test Execution)  LockProvider (PIC, Db, …)  SystemInformationProvider (e.g. Issues)  GUI Extension Points:  TestResultsViewer (E.g. Powering Server)

12. 12 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Content OverviewOngoingPlans & Visions

13. 13 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Database Migration  Previous Situation:  (-) Circuits only  (-) HWC & Sign Tests, only  (-) One Campaign Only Lsa DB AccTesting DB  Dedicated AccTesting DB (redesign):  (+) Different systems  (+) Different Tests  (+) Flexible Test Plans  (+) Many Campaigns  Done, Since CSCM, History Data migrated

14. 14 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions ELQA Integration Oracle DB (EDMSDB) Test Results + Parameters AccTesting Server AccTesting GUI Import Courtesy: M. Bednarek, J. Ludwin  Work ongoing

15. 15 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Composite Tests  One ELQA Test, seen at the circuit Level consists of one test per Magnet within the Circuit. RB.A12TST.1 MB.A12R1TST.1 MB.B12R1TST.1 MB.C12R1TST.1...TST.1

16. 16 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Notifications  Precondition for ELQA Tests  Grouping/Aggregation wrt different conditions (Systems, Tests ….)  Currently manually triggered  Later: Triggered automatically  Work ongoing

17. 17 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions System Relations & Information  Key Concept: Use existing Sources!  Possible Cooperation between TE-MPE-MS software team and BE-CO-DO (S. Jensen, D. Csikos). System Information Server System Providers: Lsa Layout Db … Relation Providers: Layout Db Runtime info (JMS) Java code analysis … Information Providers: Status Issues (Jira) Deployments Faults? … Clients (e.g. AccTesting, Diamon, ….)  Prototype in place, Work ongoing

18. 18 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Test Analysis  Vision:  Simple description of test-expectations (assertions) on signals resulting from tests (Java).  Universal GUI components to visualize problems.  See Jacek’s presentation  Work ongoing; soon: ready to write and verify complete modules

19. 19 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Data Analysis Vision  More general Data Analysis Framework  Key Concept: Perform analysis as close as possible to the data!  (Potential) Collaboration with BE-CO-DA to optimize resources, avoid duplication of efforts Analysis Framework (Cluster?) Logging DB Hadoop Cluster? Post Mortem Clients (e.g. AccTesting, Timber, Custom Analysis….) Others?  Prototype in place (Not Operational)

20. 20 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Content OverviewOngoingPlans & Visions

21. 21 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Integration of MPS Commissioning  Currently: Sharepoint Sites  Disadvantages:  Order not enforced (scheduling done ‘on the fly’).  Nothing enforces the tests to be really done.  No real overview, what was done, what still has to be done?

22. 22 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Migration of MPS Commissioning Proc.  Transform all the MPS comissioning steps to AccTesting (simple sign-only tests).  Replace one by one with automated tests.  Automate whatever can be automated! (Avoid Errors, Reproducibility)  First steps: Concentrate on systems, where many interlocks of same type:  Collimators  Vacuum  BLMs  Soon!

23. 23 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Barriers  When the commissioning of a system reaches a Barrier Point, it has to wait until all other systems (which have the same barrier) reach it also. QPSIST1 BLMIST2IST3IST4INJ1INJ2 BISIST5IST6IST7INJ3 LBDSIST8INJ4 RQTL…IST10IST11POW1POW2 Ready for Powering Ready for Injection

24. 24 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions BIC Communication Tests  Testing communication between PICs, WICs and FMCMs to BICs.  Migration of existing GUI + dedicated Db Tables to Generic Framework + DB  Reusage of existing Code. Courtesy: I.R. Ramirez  Within LS1

25. 25 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Editing of Test Plans  Currently:  Database hacks to change the test plan  New Db Schema  Even more complicated  More Test Types, more systems  Unacceptable  Has to provide at least:  Creating campaigns  Activating/Deactivating Tests  Change Test Relations/Properties (Phases, Barriers, Composite Tests, Notifications)  Soon!

26. 26 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Interlocks based on Test Plan?  Examples:  Prohibit, that circuit can be powered, before all the tests are done.  Do not allow injection of beam before all necessary MPS tests are done.  Possibly: Enforce Tests after changes:  E.g. Which tests have to be executed, after a change of a QPS – card?  Pre-defined test plans or (partially) auto-generated?   Knowledge about System-Relations required!  What to interlock?  E.g. Additional Circuit lock on PIC level which allows tests?  Injection interlock?  ?  ?

27. 27 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Custom Reports  To be used for:  Notification  Status reports  Periodical reports  Generation of Procedure Documentation  Combination with analysis language?  Export to different Formats (e.g. html, pdf, xml, csv….)  Various Libraries under investigation  ?  ?

28. 28 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Summary  AccTesting operational since 2012.  Done:  Db migrated  Ongoing:  Analysis Language  ELQA Integration  System Relation Management  Plans & Visions:  MPS Commissioning Integration  Editing of Test Plans  Reporting

29. 29 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Thank you for your Attention! Questions?

30. 30 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Data Analysis  Mandate in BE-CO-DA (C. Roderick) to provide Data Analysis Solutions  Currently in Use Case Analysis Phase  Aim to be as easy as possible for users to specify:  What to produce for an analysis case (e.g. signal names, units, description)  What are the inputs (signals and sources)  Which algorithms should be applied  When analysis should be performed (event driven e.g. fill / beam mode, time interval driven e.g. every 15 minutes)  Without user needing to:  Configure a specific development environment  Care about compilation and deployment  Look elsewhere for signal names etc. i.e. provide directory of signals and sources  Key concept: Perform data analysis as close as possible to the data!

31. 31 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions System Dependencies  Vision: Central Instance (Server?) to answer questions like:  Which magnets are connected to a certain power converter? (AccTesting – ELQA)  What is the common point for a set of crates? (Diamon)  What systems have to be potentially re-tested if e.g. a QPS card is changed? (AccTesting)  What systems are potentially affected by a OS upgrade on a certain host? (SUWG)

32. 32 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Preconditions Before: Only email exchange, that e.g. “Cryo is ready for a certain group of circuits”. Hard to track (History) No real constraint on test execution (  Many wrongly started tests in previous years) Now: ‘SignOnlyTest’ for each precondition. Automated Tests

33. 33 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Constraints Simple Extension Point, which has to decide if one Test is allowed to be run together with another one. (Simple Yes/No decision) Used to formulate requirements like: “Only start one test on one of the four circuits on the same QPS controller.” Checked by the Scheduler, to decide if a certain test-configuration is allowed or not.  See Michael’s presentation.

34. 34 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Test Step Handlers Responsible for executing a specific TestStep (execution, analysis) for a certain type of tests. Is itself responsible for communicating with other systems, if required. Examples: HwcTestExecutionHandler: Communicates with HWC Sequencer to execute the tests DaemoneAnalysisHandler: Communicates with LabView system to retrieve analysis results … Future: PicBicTestExecutionHandler?

35. 35 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions AccTesting Concept Test-Phases. Tests

36. 36 Kajetan FuchsbergerTE-MPE TM, 2013-04-25, The AccTesting Framework – Overview & Visions Test Phases Tests within a test phase can be executed in arbitrary order. TestPhase contains one or more Tests