1. SLAC FECC-III INTEGRATION ISSUES Eric J. Siskind June 12, 2003 Stanford Linear Accelerator Center

2. SLAC E. Siskind, June 12, 20032 Project Goals Replace Multibus-I micro hardware with commercial-off-the-shelf personal computers. Replace special purpose accelerator networks (SLCnet, KISnet, PNET) with a single commercial-off-the-shelf TCP/IP network such as switched gigabit Ethernet. Develop personal computer CAMAC interface to replace MBCD/MBCD-II.

3. SLAC E. Siskind, June 12, 20033 New CAMAC Features CAMAC interface differs from MBCD/MBCD-II in four major ways: –Two board design to support remote I/O; –Real-time preemptive hardware with recovery; –Maximizes CAMAC utilization when user software environment in PC is multi-threaded; –Large (i. e. comparable to existing micro) user programming environment within interface.

4. SLAC E. Siskind, June 12, 20034 Hardware Architecture Hardware consists of two circuit boards connected by up to 10 km of single mode fiber optics: –PCIL (“PCI Link”) board plugs into PCI option slot in PC; –FECC (“Front-End CAMAC Controller”) is now a stand-alone chassis with a single-width CAMAC module to access PDU triggers. PC moves from micro’s current location to MCC computer room, leaving behind only FECC.

5. SLAC E. Siskind, June 12, 20035 PCIL Features Basically an intelligent special-purpose Network Interface Card. Processor with DMA PCI block transfer master and DMA fiber optic link transmitter/receiver. Makes no assumptions about nature of PC’s real- time operating system. Semi-static assembly language PROM code with downloaded updates. First generation hardware doubles as Alpha SLCnet interface.

6. SLAC E. Siskind, June 12, 20036 PCIL Code Functions Supports PC  FECC message passing. Supports FECC remote read/write access to PC memory with optional PC interrupt. Performs function-specific scatter/gather DMA to/from PC memory for all classes of operations, avoiding unnecessary buffer copying by PC CPU. Forwards trigger pattern from PCI registers to FECC (only in 1 st generation; pattern forwarding in hardware in 2 nd generation).

7. SLAC E. Siskind, June 12, 20037 FECC Features Logical successor to MBCD/MBCD-II. Accesses host memory via point-to-point fiber optic link instead of parallel I/O bus. 1 st generation supports MBCD-II’s four strings of SLAC serial CAMAC crate controllers (5 MHz, bit serial, half duplex). 2 nd generation adds support for four strings of IEEE standard crate controllers (5 MHz, byte serial, full duplex) and dual Bitbus strings.

8. SLAC E. Siskind, June 12, 20038 CAMAC Interrupt Recovery & Control If a CAMAC block transfer is interrupted, hardware can repeat the previous write operation (address pointer load?) with updated write data before resuming the transfer. Hardware can prevent an interrupt between CAMAC cycles transferring two 16-bit halves of a single 32-bit word. Hardware can prevent an interrupt of a particular block transfer entirely (needed because of a current PIOP feature).

9. SLAC E. Siskind, June 12, 20039 System Performance Many PC jobs now can have simultaneously outstanding long CAMAC packages. All packages are sent to FECC firmware ASAP, and parsed into their individual (block transfer) operations in a single crate (“packets”). Packets are immediately queued to hardware for the cable accessing the target crate. Cable operation begins as soon as any outstanding package has a packet requiring access to a crate on that cable. Parallelism now achieved over multiple packages.

10. SLAC E. Siskind, June 12, 200310 PCIL2 Intelligence Analog Devices ADSP-21060 “SHARC” 40 MHz 32-bit DSP on 2 nd generation PCIL. 40k x 48 bit on-chip program memory; 64k x 32 bit on-chip data memory. 256k x 48 bit external program/data memory. 256k x 64 bit external DMA I/O data memory.

11. SLAC E. Siskind, June 12, 200311 FECC3 Intelligence IBM PPC405 250 MHz 32-bit PowerPC embedded in Xilinx FPGA on 3 rd generation FECC. 16k x 64 bit on-chip program memory; 16k x 32 bit on-chip data memory. 2048k x 64 bit external program/data memory with DMA link/CAMAC hardware.

12. SLAC E. Siskind, June 12, 200312 User FECC Programming Real-time executive plus MBCD/Bitbus emulation uses ~15% of on-chip program memory, ~25% of on-chip data memory, and ~5% of off-chip RAM. Remaining minimum of 15+ megabytes of memory on 250 MHz 32-bit processor exceeds capacities of existing 386/486 CPU boards. Move existing 360 Hz interrupt driven code from micro to CAMAC interface to minimize CAMAC access delay. C run-time environment with dynamic heap allocation; full environment and stack preserved over context switch.

13. SLAC E. Siskind, June 12, 200313 2 nd Generation Hardware PCIL2 uses 64 bit/66 MHz PCI slot. Single specialized fiber optic link for PCIL2  FECC3 communication. 125 megabyte/second full duplex link, but 9/19 used for hardware overhead. All peripherals FPGA-based DMA except for Bitbus (programmed I/O). Hardware based on one large FPGA per board, plus SHARC on PCIL2.

14. SLAC E. Siskind, June 12, 200314 2 nd Generation Status PCIL2 now debugged and tested. Two PCIL2 boards built in 2001; three more in 2002. FECC3 design in completed in 2002; FPGA delivery currently 29 August (5 month delay). Initial build of two FECC3 boards in 2003; three more after debugging.

15. SLAC E. Siskind, June 12, 200315 Bitbus Master Existing master based on MCS-51 2 MHz 8 bit microcontroller with SDLC/HDLC serial link. 8044BEM is an MCS-51 with mask-programmed firmware (iDCX-51 RTE plus Bitbus application). FECC3 has dual-channel programmed I/O hardware (one PowerPC FIFO read or write per 32 bits) to send a packet to one slave and receive that slave’s response. Poll list, counters, etc. maintained by new PowerPC code in FECC3. Enhanced real-time support via reserved high priority buffers and multiple prioritized queues.

16. SLAC E. Siskind, June 12, 200316 Schedule Migration of 360 Hz processing in progress (TEG). All optimizations not necessary at early stage. Trying to deploy MPG plus one ordinary PC-micro in development system in January, 2004. Hoping to replace MPG plus one ordinary micro in summer of 2004.

17. SLAC E. Siskind, June 12, 200317 Simplified Hardware Configuration No PC  PC fast feedback (KISnet equivalent) network traffic. Use 2 nd generation PCIL-FECC hardware: –Include IEEE CAMAC and Bitbus; –Need PC with 64 bit/66 MHz PCI slot. MPG  PC (PNET equivalent) pattern broadcast on dedicated point-to-point Ethernet link. PC-based MPG (MP10/MP11) broadcasts pattern to existing micros via FECC3 BITbus interface reconfigured as a PNET transmitter; talks to SP00/SP01 and BIC via PC  VME reflective memory links.

18. SLAC E. Siskind, June 12, 200318 Integration Projects Build, boot, and debug iRMX-III in off-the- shelf personal computers. Provide minimal FECC debugging support. Make real-time micro  micro data communications streams coexist with slower back-end  front-end traffic on switched TCP/IP network (deferred). Move 360 Hz interrupt processing from micro to FECC (MPG + pattern consumer).

19. SLAC E. Siskind, June 12, 200319 Integration Projects II Change control of 360 Hz interrupt processing from shared memory to network link model. Integrate PCIL-FECC CAMAC/Bitbus emulation. Add support for real-time CAMAC/Bitbus; Modify fast feedback actuator to specify real-time priority for CAMAC/Bitbus operations. Modify slower micro jobs to send larger packages to CAMAC interface to maximize parallelism. Modify klystron job to protect PIOP CAMAC operations from interrupts.

20. SLAC E. Siskind, June 12, 200320 Integration Projects III Support logical to physical micro mapping. Support PC access to file server delivering current versions of PCIL and FECC executable images appropriate to different types of micros.