Control Systems for the APTM and GRID Joao Paulo Martins Control Systems Engineer - ICS European Spallation Source ERIC 2019-01-09

Summary Introduction; Prototype tests in Japan; Hardware project; Software project; Machine Protection interface; Conclusion;

Layer Architecture of the ESS Control Systems Introduction Layer Architecture of the ESS Control Systems

Experiment workstations other human interfaces Layer Architecture of the ESS Control Systems ESS Main Control Room Experiment workstations other human interfaces System GUIs Equipment GUIs Physics Apps Config. Tools Data on Demand PSS GUIs NICOS DMSC High Performance Data Network EPICS EPICS IOC EPICS IOC EPICS IOC EPICS IOC EPICS IOC Data on Demand EPICS Archiver Timing System Event Generator Machine Protection Fast Beam Interlock System Personnel Safety Systems PLC PLC Ether CAT MicroTCA System MicroTCA System PSS PLC Fieldbus Device signals Remote I/O Controller Device Electronics Device Electronics Controller Remote I/O Remote I/O Pump Flowmeter Motor Sensor Chopper Power Supply Switch Sensor Sensor Mech. Movement RF Proton Neutron Temp Electricity Magnetic H2O Flow He Flow etc.

other human interfaces Architecture of the APTM/Grid Control Systems ESS Main Control Room other human interfaces System GUIs Equipment GUIs Physics Apps Config. Tools Data on Demand High Performance Data Network EPICS EPICS IOC EPICS IOC Data on Demand EPICS Archiver Timing System Event Generator Machine Protection Fast Beam Interlock System Personnel Safety Systems Ether CAT MicroTCA System Fieldbus Device signals Controller Device Electronics Motor Sensor Mech. Movement Temp Current

Summary Introduction; Prototype tests in Japan; Hardware project; Software project; Machine Protection interface; Conclusion;

Prototype tests in Japan

Prototype tests in Japan Installation and remote operation of the APTM/Grid prototype at J-PARC using as much as possible “ESS environment”; MicroTCA as main component: DAQ and CPU; Beckhoff EtherCAT for motion control and temperature reading; EPICS for operation and data collection; Lot of experience gained; Robustness of the MicroTCA crate confirmed; Motion Control IOC interface and operation needs refinements; EPICS IOCs instances should be easily manageable; Timing Events MicroTCA System APTM blades Digitizer Timing Motion Control Software Control System Software APTM grid External clients Motors / encoders EtherCAT I/O Modules

Summary Introduction; Prototype tests in Japan; Hardware project; Software project; Machine Protection interface; Conclusion;

Hardware Project FMC-PICO-1M4 selected as digitizer; 20 bits current measurement within 1 uA range @ 1 MSPS; 4 channels per FMC; IOxOS IFC1410 FMC carrier; ICS supported platform; No support for CAENELS FMC-PICO-8 yet. [On-going activity with Lodz in-kind]; Needs custom firmware implementation for interlock functions; Concurrent Technologies MTCA CPU; Based on 4-core Intel i7 processor (new version based on Intel Xeon improves performance); Front-panel Ethernet ports enables its use as EtherCAT master; MRF MTCA-EVR-300 Standard timing component for MicroTCA crates, fully integrated in EPICS;

Hardware Project Preliminary proposal of MicroTCA equipment: One 12-slot 9U MicroTCA crate for all 4 APTM units: 10 slots for digitizer unit, 2 slots for CPU and Timing: 80 analog inputs; One 6-slot 3U MicroTCA crate for the GRID: 4 slots for digitizer unit: 32 analog inputs; Alternative proposal: Separate APTM hardware in different crates; Offload CPU consumption (although maybe only one is enough); Local CPUs of the IFC1410 can also be used for data acquisition;

Hardware Project Motion Control Hardware Beckhoff EtherCAT modules; Motion Control Hardware Standards: https://confluence.esss.lu.se/display/MCAG/Motion+control+electronics EtherCAT master can be implemented using regular Network Interface Card: MicroTCA CPU; Motors / encoders

Summary Introduction; Prototype tests in Japan; Hardware project; Software project; Machine Protection interface; Conclusion;

Software Project EPICS IOC for data acquisition and operation control have access to the digitizer and timing cards via PCIe drivers; Main functionalities: configuration of the timing triggers, analog data acquisition, beam profile measurement (GRID); EPICS device support layer implemented based on areaDetector framework; areaDetector will be the common framework for digitizers in EPICS at ESS; Support for IOxOS cards in areaDetector currently under development; Single IOC to control timing and data acquisition. Motion Control runs in separate IOC for now – Scenario with single IOC for all systems will be evaluated;

Software Project External clients APTM/GRID IOC Beam profile measurement; APTM blades measurement; Acquisition triggers configuration; Motion Control IOC Motor record; Sequencer; Temperature reading; AreaDetector TimeSeries Plugin AreaDetector ROI Plugin AreaDetector Stats Plugin Other AD plugins MRFIOC2 Device Support ECMC Open-Source Motion Control Framework areaDetector (asyn) device support for IFC1410 Developed at ESS TSC kernel driver MRF kernel driver Etherlab EtherCAT master driver PCIe PCIe EtherCAT

Software Project EPICS Environment EEE -> E3 Same concept as EEE (dynamic loading of shared libraries) but different implementation; E3 build system can accept external code “as it is”; Strict dependencies: integrator should specify which versions will be used in compilation and IOC startup. No automatic loading anymore; Standard set of EPICS extensions already supported: autosave, iocStats, recSync (Channel Finder). Standard template for a all these IOC services; procServ and systemd services used to run IOCs – currently under tests; EPICS 7: version 7.0.1.1 already supported in E3. The new “qsrv” engine directly maps the DBR types (used in EPICS 3 database files) to new Normative Types; For more complex data structures – structured PVs – changes on database files are needed; BI requested a standard template of structured data for the waveforms acquired in digitizers systems: to be done;

Summary Introduction; Prototype tests in Japan; Hardware project; Software project; Machine Protection interface; Conclusion;

Machine Protection Interface Fast Beam Interlock System (FBIS) Interface: Data process on the FPGA should trigger interlock functions; How to do this? FFT? Integrator? Envelope detection? Bad condition should be detected within 10 µsec? Signals are divided in different digitizer cards. AMC-to-AMC backplane communication needed – avoid CPU; Support for FMC-PICO for IOxOS FPGA design kit should be relatively simple. Currently under development by in-kind partner (Lodz); FBIS interface under development: MTCA RTM – to be available in September-19; IFC1410 AMC Carrier FBIS RTM FBIS SCU FPGA

Summary Introduction; Prototype tests in Japan; Hardware project; Software project; Machine Protection interface; Conclusion;

Evaluation of the requirements Current design evaluation Depends on custom FPGA design to implement the measurement and interface to FBIS. Should be verified by extensive tests, but 10 microseconds is not a big challenge for a FPGA;

Evaluation of the requirements Current design evaluation Already verified with different IOCs configurations: Struck, AMC-PICO-8, IFC1410 – areaDetector or NDS3-based; FMC-PICO sampling rate is 1 MSPS. Waveforms for every pulse can respond at 14 Hz on EPICS level; IFC1410 has 2 banks of 512 Mbytes. Considering 8 channels @ 1 MSPS (32 bits per sample), we have room to store ~ 16 seconds continuously; Should be verified; Sampling at 1 MSPS gives us ~ 100 intrapulse samples. Averaging is allowed; Sampling at 1 MSPS gives us ~ 5 intrapulse samples. Averaging is allowed;

Conclusion Hardware project (based on IFC1410) still needs some firmware implementations. Integration of the FMC-PICO can be demonstrated until Jun-19; FBIS functionalities still at an early stage; Interlock conditions can be simulated and tested with current FPGA platforms. Vertical integrated tests with FBIS will need more time – not achievable in Jun-19; ICS should carry on performance tests with different CPU/AMC configurations to support the decision of how many crates will be used; Control system software (EPICS IOCs) design already tested in J-PARC; No major changes foreseen – areaDetector as main component; EPICS 7 IOC should be the goal – achievable by Jun-19;

Conclusion Obrigado!