1. NSLS-II Fast Orbit Feedback with Individual Eigenmode Compensation Yuke Tian, Li Hua Yu Accelerator Division Photon Science Directory Brookhaven National Lab EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

2. Outline NSLS-II orbit feedback system Technical requirements and specifications NSLS-II site field vibration measurement Slow and fast corrector locations Fast orbit feedback with individual eigenmode compensation Typical fast orbit feedback algorithm NSLS-II FOFB algorithm with individual eigenmode compensation Comparison of the two algorithms Solving the calculation problems from two directions NSLS-II fast orbit feedback implementation Summary EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

3. Energy 3.0 GeV Circumference 792 m Number of Periods 30 DBA Length Long Straights 6.6 & 9.3m Emittance (h,v) <1nm, 0.008nm Momentum Compaction.00037 Dipole Bend Radius 25m Energy Loss per Turn <2MeV Energy Spread 0.094% RF Frequency 500 MHz Harmonic Number 1320 RF Bucket Height >2.5% RMS Bunch Length 15ps-30ps Average Current 300ma (500ma) Current per Bunch 0.5ma Charge per Bunch 1.2nC Touschek Lifetime >3hrs NSLS-II technical requirements & specifications EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

4. NSLS-II site field vibration measurement EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

5. Slow and fast corrector locations Even Cell Odd Cell EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

6. Let’s design a FOFB system EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 BPM PS Controller Orbit feedback calculation BPM data Tx/Rx BPM data around the ring Tx/Rx BPM data around the ring Corrector setpoint BPM PS Controller Orbit feedback calculation BPM data Corrector setpoint

7. Can we make a clean architecture ? EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 1. Try to deliver BPM data to a place that orbit calculation module have directly access. Don’t shove data around too much. 2. Similarly, try to deliver corrector setpoint from a place that orbit calculation module have directly access. 3. Looks we need a place that can:  Receiver local BPM data;  Tx/Rx BPM data to/for other cell;  Carry out FOFB calculation;  Tx corrector setpoints to PS control system. Cell Controller !

8. Controller R -1 =VΣ -1 U T Accelerator R=UΣV T Compensator (PID etc) R -1 : reverse response matrix R: response matrix Typical fast orbit feedback algorithm The ill-conditioned response matrix will cause numerical instability. Solution: 1) Truncated SVD (TSVD) regularization 2) Tikhonov regularization EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

9. Each of the corrector setpoint is calculated from a 1xM vector and Mx1 vector multiplication. If 8 BPM/cell in NSLS-II, M=240. Total: ~ 240 MAC. For each corrector plane Typical fast orbit feedback algorithm EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

10. NSLS-II FOFB algorithm – compensation for each eigenmode Fast orbit feedback system is a typical multiple-input and multiple-output (MIMO) system. For NSLS-II, there are 240 BPMs and 90 fast correctors. The BPMs and correctors are coupled together. One BPM reading is the results of many correctors. One corrector kick can also affect many BPM readings. It is difficult to design a compensator for all noises with different frequencies. It is desirable if we can decouple the BPM and corrector relationship so that the MIMO problem can be converted into many single input single output (SISO) problems, for which control theory has many standard treatments. Fortunately, SVD already provides a solution: it projects the BPMs input into the eigenspace, where each component is independent. We can design many SISO type compensators (one for each eigenmode) and apply the standard SISO control theory to treat each eigenmode problem in frequency domain without affecting other eigenmodes. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

11. c 1, c 2, …, c N is the input projections in the eigenspace. Q 1 (z), Q 2 (z), …, Q N (z) is the compensator for each eigenmode. We want to prove that Q 1 (z), Q 2 (z), …, Q N (z) only corrects the corresponding eigenmode in eigenspace without affecting other eigenmodes. UTUT Accelerator R=UΣV T Q(z) Σ -1 V NSLS-II FOFB calculation – compensation for each eigenmode EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

12. UTUT Accelerator R=UΣV T Q(z) Σ -1 V In cycle n, The pure effect of Qi(z) is on the ith eigencomponent. The noise signals are also decoupled into the eigenspace. This gives us freedom to suppress the noises in eigenspace using SISO control theory. Since, NSLS-II FOFB calculation – compensation for each eigenmode EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

13. Decpoupling or not ? Calculation compare Calculation AmountWithout DecoupingWith Decouping One corrector strength (one plane) M+k =243N(M+N+k)= 29970 90 corrector strength (one plane)N(M+k) = 21870N*N*(M+N+k) = 2697300 For each corrector plane, M multiplication and accumulation(MAC), followed by k MAC for corrections. Calculation (one plane): One corrector : M+k N correctors: N*(M+k) Without decoupling: With decoupling: It takes NxM MAC to decouple the inputs into eigenspace. It takes N*k MAC for N compensators in each eigenmode. It takes N*N to get one corrector strength. Calculation (one plane): One corrector: N*(M+N+k) N correctors: N*N*(M+N+k) Assume N=90, M=240, k=3: Assume 50us of 10Khz FOFB time is used for the caluclations: N*N*(M+N+k)/50us=5.400 GMAC/s. For doing two plane FOFB in 10us: 5.4*5*2= 54 GMAC/s The high end DSP chip gives about 200-300 Millions floating point operations (MFLOP). EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

14. Solving the calculation problems from two directions Distributed FOFB system: two tier communication 30 cells in the storage ring All the BPM data is delivered to all the 30 cell controller within 12us, cell controller only needs to calculate the local corrector strength. This distributed architecture reduces the calculation by factor of 30. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

15. Solving the calculation problems from two directions FPGA’s powerful DSP performance FPGA ‘s DSP power is mainly gained through the parallel computation capability. The parallelism increases FPGA DSP power (vs genetic DSP or CPU) by a factor of more than 2000 (Virtex 6). Standard DSP/CPU process: sequential FPGA’s fully implementation One of the DSP48 block in FPGA EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

16. Solving the calculation problems from two directions FOFB calculate in FPGA: UTUT Accelerator R=UΣV T Q(z) Σ -1 V U1TU1T U2TU2T UNTUNT V1V1 VNVN V2V2 Θ N Θ 2 Θ 1 Q 1 (z) Q 2 (z) Q N (z) NSLS-II FOFB Model EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

17. Solving the calculation problems from two directions FOFB calculate amount in FPGA: U1TU1T U2TU2T UNTUNT V1V1 VNVN V2V2 Θ N Θ 2 Θ 1 Q 1 (z) Q 2 (z) Q N (z) Decoupling into eigenspace: All components c 1 (n), c 2 (n), …, c N (n) is calculated in parallel (M MAC) Compensation for each eigenmode: The k step compensations are done in parallel. (k MAC) Corrector strength calculation: All corrector strength is calculated in parallel: N MAC The total calculation is reduced to: M+N+k = 240 + 90 + 3 = 333 MAC. FPGA will finish it within a few us. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

18. NSLS-II FOFB Implementation U 1 T V1V1 Θ N Θ 2 Θ 1 Q 1 (z) Q 2 (z) Q N (z) RAM BPM data From SDI link Control System (EPICS IOC) Download through GigE interface RAM MUX RAM U 2 T U N T RAM MUX RAM MUX RAM MAC RAM MUX RAM V2V2 MUX RAM VNVN MUX RAM To PSC EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

19. Floating point calculation vs fixed point calculation: FPGA computation is usually fixed point calculation, the quantization errors should be carefully controlled so that the calculation is accurate. NSLS-II FOFB Implementation FPGA fixed calculation block Matlab floating point calculation EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

20. NSLS-II FOFB Implementation Decoupling calculation error Error with PID calculation for one eigenmode EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

21. Status EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 The hardware design (PCB, chassis) for BPM, cell controller and PSC are all done. PSC is in production stage. BPM and cell controller are in the first article of production stage. BPM and PSC FPGA firmware, EPICS drivers and database development are all done. All cell controller blocks (SDI, FOFB etc) are all. The cell controller integration is in progress. Since we have the fast fiber SDI to deliver data around the ring, cell controller’s SDI link will also be used as fast machine protection system that deliver critical system (such as the vast valve signal from vacuum system) around the ring within much less than 1ms. This latency is impossible for PLC to achieve.

22. Open source hardware EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 Want it ? You got it.

23. Open source hardware EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 We like to open the hardware we designed at BNL to the community. These includes all the PCB design, the FPGA firmware design and the EPICS IOC (driver, database) design. We welcome any suggestions and cooperation from the community. PSC PSI

24. Open source hardware EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 The digital front end (DFE) board (Virtex6 FPGA) can be used as a common platform for many systems since it provides the basic common elements such as large memory (1 GB DDR3), embedded CPU, Gigbit Ethernet port to EPICS IOC, high speed SDI link(use 2 of the 6 available SFPs), and many user IOs (>160) to the user daughter board. BPM Cell Controller Share the same DFE

25. Open source hardware EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 BPM control panel and raw ADC data

26. Open source hardware EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 BPM turn-by-turn data and 10KHz FOFB data

27. Open source hardware EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011 BPM 10Hz slow acquisition data

28. NSLS-II’s stringent emittance requirements need a efficient fast orbit feedback system. The two tier communication structure and the FPGA- based fast orbit feedback calculation architecure is designed for achieve the requirments. Algorithm with individual eigenmode compensation is proposed. The typical MIMO feedback problem is converted into many SISO problems. This algorithm enables accelerator physicists to correct the beam orbit in eigenspace. We compared the calculations for FOFB with and without individual eigenmode compensation. We found that the proposed NSLS-II FOFB algorithm needs a large amount of calculations. However, benifited from NSLS-II FOFB architecure, the challenge can be conquered. We expect a successful application of the NSLS-II FOFB algorithm during the NSLS-II commissioning and daily operation. BNL like to open all the designs to the community as part of the open source hardware initiative. Summary EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

29. Acknowledgments FOFB algorithm Igor Pinayev Sam Krinsky BPM/ Cell controller development : Kurt Vetter Joseph Mead Kiman Ha Alfred Dellapenna Joseph De Long Yong Hu Guobao Shen Om Singh Bob Dalesio PSC and PS design: Wing Louie John Ricciardelli George Ganetis EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011