Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 1 Linac Controls Upgrade Status Tom Himel SLAC November 11, 2008

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 2 Outline Introduction Phasing New architecture Long lead items status Planning Switchover Summary

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 3 What a difference one letter makes I started managing the linac controls upgrade a couple of months ago. Last Friday morning, John Galayda told me I would no longer be a reviewer on the FAC. Instead I would be a reviewee. What a difference a single letter makes

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 4 Introduction Linac (and DR and PEP and ESA) use control system built in early 80’s for the SLC. PEP-II used EPICS for new functionality with some connections between SLC and EPICS LCLS used EPICS for all new devices. Provided connections so SLC high level apps could be used for new EPICS devices

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 5 Introduction (2 of 2) Through retirements and layoffs, have lost most people who know how to maintain the SLC control system. Have decided to upgrade the SLC control system to EPICS with a series of AIP projects. This will reduce chances of long downtimes due to some problem where expertise is lacking causing a long MTTR. Hardware is old. Upgrade is overdue. SLC control system has many years of applications being added and fine tuned. Main difficulty of the upgrade is to not lose too much this functionality.

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 6 SLC control system architecture

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 7 Phases Phase 1 makes LCLS independent of the Alpha and multibus micros Replaces about 15 micros with VME crates, CPUs, and CAMAC interface. 1 micro in each of last 10 sectors, few in BSY, few system micros (e.g. MPG) Main work is software Phase II replaces the CAMAC with more modern hardware Main expense is hardware Phase III-N propagates above upgrades to other regions of the accelerator complex Little planning here yet There are different types of functionality and hardware modules in other regions so not just simple replication

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 8 SLC control system architecture Phase I Phase II

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 9 Phase I Hardware Replace 15 micros with VME crates/CPUs/EVRs Develop VME to SLAC serial CAMAC interface Run Ethernet to new VME crates Replace few small systems with new hardware instead of developing new software. Expect that BPMs will be upgraded All quite straight forward

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 10 Phase I software Replace >1,000,000 lines of software on Alpha Replace >100,000 lines of software on micros Not as bad is it appears Some functionality no longer used at all (e.g. lattice diagnostics) Some functionality not used by LCLS (PEP-II applications) Much already exists in EPICS (e. g. data archiving and configurations and distributed real-time database) Some are high level applications being developed for LCLS by the HLA group It is still the major headache of phase I.

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 11 Long lead items started VME CAMAC interface prototype working Can read the position of a switch with a CAMAC module and display it on an EPICS display. Used: New VME to CAMAC hardware interface designed by Jeff Olsen. New CAMAC software drivers written by Bob Sass. New test software written by Sheng Peng. Prelimary design for simple CAMAC module support (SAM, IDIM, LDIM) done Design for RF support which has most comple module (PIOP) started a month ago. Have purchased micro TCA hardware and about to start phase II RF hardware design effort

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 12 Planning A great deal of effort has gone into documenting existing functionality and determining which parts need to be implemented in EPICS. Biggest schedule uncertainty is amount of software that must be implemented before we can switch over. Making list and will get agreement (signatures) with users. There are serious, tough decisions to make. Need to agree on some functionality that will NEVER be needed and others that can be delayed until after switch- over.

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 13 High Level Applications Most HLAs are being written by Patrick Krejcik’s group and are not officially part of the upgrade. They are essential for a successful upgrade and also determine the architecture for HLAs and mid level applications specific to the upgrade. I’m working closely with Patrick’s group.

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 14 Schedule There was a very aggressive schedule proposed early in this project. At an earlier review, reviewers were concerned that 9 FTE-years was a significant underestimate of the software work. Also concerned that that project start will be slow because manpower will continue to be used to make LCLS controls work well. Plan to take considerably more time. Maybe 2 years to complete phase 1 and finish the R&D for phase 2. Still working on more detailed schedule. Concentrating on documenting scope of the project. Controls is training people to keep the SLC control system running Will be risk of significant unscheduled downtimes due to obscure problems where expertise is lacking.

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 15 Switchover Thorough testing will be done in sectors 17 and 18 (not used by LCLS). Via a PV gateway and aliases HLAs will support devices both before and after switch-over. Can switch a sector at a time instead of everything at once. Will need some scheduled LCLS downtime to test and implement the switchover.

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 16 Planning – task list

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 17 Worries These are mostly not under my control so lab management needs to be aware of them. 1.As implementation of phase I will take longer than originally planned and as expertise in the SLC control system is needed until the rest of the linac, damping rings, NLCTA, the gun lab, and the test lab are converted, people experienced with the SLC control system must be kept or new ones trained. 2.If FACET proceeds on its schedule of completion in Oct 2009, it clearly will not use the new control system and that much more expertise in the SLC control system will need to be maintained. If FACET proceeds on a slower schedule, management will need to decide soon whether a controls upgrade is done as part of the project, or with operating or AIP funds before the project, or if it will use the SLC control system.

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 18 Worries 3.This is a large, important project. Even so, many LCLS needs discovered during the commissioning process will be higher priority. It is better to do this project right and, if necessary, slowly, than to rush it to meet a schedule deadline and lose too much functionality when it is turned on. 4.There are 2 steering feedback in the region being converted. If they need to run at 120 Hz, then the EPICS version of 120 Hz feedback must be done before the switchover. (Planned for Jan 2010.)

Tom Himel Linac Controls Upgrade Nov SLAC National Accelerator Laboratory 19 Summary Linac upgrade is needed Can survive while we work on the upgrade Intense work on the upgrade is awaiting personnel being freed from LCLS project completion and commissioning Plans are progressing but still need detailed schedule which is in progress.