GlueX Collaboration Meeting 12GeV Trigger Electronics 10 May 2010 R. Chris Cuevas 1.FY10 Project Goals  Update from January 2010 Collaboration Meeting 2.Hardware Design Status Updates 3.Summary NIM

FY10 Project Goals Baseline Improvement Activities (BIA) have been established for FY10  FADC250 Rev- BIAFAD Revision includes 12bit ADC and consolidation of FPGA which will reduce part count significantly. Use latest Xilinx technology and include new requirements.  F1-TDC Rev-2 BIAF1T Revision includes 48 channel ‘mode’ for FDC, add VXS signaling, upgrade FPGA, add event RAM, and pulse output feature.  SSP (Prototype) - BIATRG Complete schematic, board layout, assembly, and firmware for at least two units.  GTP (Prototype) - BIATRG Complete schematic, board layout, assembly and firmware for a single unit.  Complete VXS crate specification and procurement for ALL VXS crates needed for 12GeV applications. Allows for quantity price reduction and ‘phased’ delivery during installation period. 2

Hardware Design Status SubSystem Processor (Prototype by end of June 2010!!) Ahead of schedule Board fabrication files checked and ready for order Components and assembly order prepared Firmware development on track Global Trigger Processor (Prototype by end of FY10 will slip,,,) Specification has been updated and latest proposal presented and reviewed Schematic not started Schedule dependent on additional EE (Position open) Crate Trigger Processor (No work plan for FY10) Two prototypes successfully tested! (FY09) Final revision in FY11 work plan Signal Distribution switch (Revisions started – Welcome to Nicholas Nganga) Two prototypes successfully tested! (FY09) Final revision has been started and will extend into FY11 work plan 3

Hardware Design Status Trigger Interface/Trigger Distribution (Latest revision by end of FY10 ) Work activity schedule created ( William Gu; Ed Jastrzembski) Initial version of TI/TD successfully tested FY09 Latest revision specification document has been updated Latest revision will be a single board design with TI/TD functionality Schematic capture complete Component placement complete Board routing complete Components ordered Firmware development on track Trigger Supervisor (FY11-12 work plan) Specification has been created and reviewed Detailed work activity schedule will be created 4

Specification Status VXS powered card enclosures ( Procurement by end of FY10 ) Crate specification has been created and ordering strategy discussed with procurement department. Multi-year procurement with delivery quantities per year to be determined. Specification completed and Request For Information (RFI) has been prepared. Accounts have been adjusted to reflect multi-year order and delivery schedule to reflect installation needs for both Halls D & B. Trigger System Fiber Optics (FY11 work plan) Draft system diagram has been specified Final component specifications will need to be completed and ordered in FY11 5

SSP Layout Status SSP PCB Layout 100% - will be sent out for assembly in the next week or so Assembly should arrive late May, module testing in June Initial firmware is complete for multi-crate trigger testing (currently just energy sums) Ben Raydo 6

VME64x P2VXS P0VME64x P2 Jtag DDR2 SO-DIMM (up to 4GBbyte) VME Address DIP Switches Power Supply Margin Header Status LEDs 2x Bi-Directional NIM Ports 4x LVDS/PECL/ECL Inputs 4x LVDS Outputs 8x 10Gbps Fiber Transceivers SSP PCB Module Overview Ben Raydo 7

Virtex 5 - FX70T - 4x 10Gbps Fiber Transceiver -Reed Solomon FEC Decoder - 500MB/s FX70T LX50T Bus -Used for VME Bridge - 4x 5Gbps FX70T->LX50T Bus -Used for L1 Streams HFBR Channel Full Duplex 2.5Gbps Fiber Streams HFBR Channel Full Duplex 2.5Gbps Fiber Streams HFBR Channel Full Duplex 2.5Gbps Fiber Streams HFBR Channel Full Duplex 2.5Gbps Fiber Streams HFBR Channel Full Duplex 2.5Gbps Fiber Streams HFBR Channel Full Duplex 2.5Gbps Fiber Streams HFBR Channel Full Duplex 2.5Gbps Fiber Streams HFBR Channel Full Duplex 2.5Gbps Fiber Streams Virtex 5 – LX50T - 2x 10Gbps VXS Switch Links - VME64x Interface - 500MB/s FX70T LX50T Bus Used for VME Bridge - 8x 5Gbps FX70T->LX50T Bus Used for L1 Streams -DDR2 SO-DIMM Used for L2 dictionary DD2 SO-DIMM -Up to 4GB Memory Module - <60ns Random Access Time - 3.2Gbyte/sec burst rate VME64x - Support for A24/A32, D32/D64 Transactions -Single Cycle, BLT, 2eSST Support VXS-P0 Receives: CLK250,TRIG1,TRIG2,SYNC,LINKUP Transmits: TXDATA[3..0],Busy Front-Panel I/O TX: 4x LVDS RX: 4x AnyLevel (LVDS,ECL,PECL) Bidirectional: 2x NIM Virtex 5 - FX70T - 4x 10Gbps Fiber Transceiver -Reed Solomon FEC Decoder - 500MB/s FX70T LX50T Bus -Used for VME Bridge - 4x 5Gbps FX70T->LX50T Bus -Used for L1 Streams 10Gbps L1 Stream4Gbps VME Bridge 4x 5Gbps L1 Streams 10Gbps L1 Stream 2x10Gbps L1 Streams Clock Distribution -Distributes to all FPGAs: 50MHz, 250MHz Clocks -Local 250MHz & 50MHz -SMA 250MHz, VXS-P0 SWA or SWB SSP Block Diagram From CTPs (Crate-Trigger-Processors) To GTPs (Global-Trigger-Processors) Ben Raydo 8

Subsystem Processor: L1 FCAL (11) BCAL (16) Tagger (2) ST (1)TOF (2) L1 Subsystems (# Crates) Energy Hit Pattern 10Gbps fiber optics Sub-System-Processor (SSP) consolidates multiple crate subsystems & report final subsystem quantity to Global-Trigger-Processor (GTP) 32bit quantity every 4ns Subsystem Energy Sum & Hit Pattern (10Gbps to GTP) Global Trigger Crate: 9

8 Optical Transceivers HFBR-7924 TrgSv Rev. 2 interface External I/O VME  PROM (FPGA firmware) Emergency/remote re-programming VXS P0 TD mode: from SD TI/TS mode: to SD VME 64x One dedicated link for redundant data collection Trg/Clk/Sync outputs On row_C Trigger Interface & Distribution Xilinx Virtex-5 LX30T-FG665 William Gu 10

TID Trigger/Clock/Sync distribution/Busy Two sided, 12-layer VME board (6Ux160mm) PCB layout finished PCB fabrication after independent review/check FPGA firmware development underway Board testing in the summer. (Virginia summer is earlier) Module has been designed to function as Trigger Interface or Trigger Distribution Module can be configured to function as a Trigger Supervisor (Trigger signals received on front panel I/O connector) VXS payload board format Provides communication from VME to SD & CTP modules via I^2C serial links. William Gu 11

TID Test/Commissioning setup: Up to 9 crates Can be supported TD with TS function TI#1 TI#2 TI#8 SD#1 SD#2 SD#8 FADC#1-16 Fiber VXS SD#9 VXS William Gu 12

Level 1 & Trigger Distribution Up to 128 front-end crates CPU VXS-Crate Global Trigger CrateTrigger Distribution Crate Trigger Decisions L1 Subsystem Data Streams (hits & energy) Fiber Optic Links SSP can manage 8 CTP (Crates) Front-End Crates VME Readout to Gigabit Ethernet TI SSP GTP SSP CPU VXS-Crate TS TD SD Clock Trig1 Trig2 Sync Busy Fiber Optic Links TD can manage 8 TI (Crates) 13

Global Trigger Processor: (GTP) SSP up to 8 SSP Modules (64 L1 Front-End Crates ) GTP SSP GTP Each Connection: 8Gbps SSP->GTP (32bits every 4ns) Up to 8x L1 Crates per SSP Tagger (Hits) Start Counter (Hits) TOF (Hits) BCAL (Energy) FCAL (Energy) from CTPs Trigger Decisions to Trigger Distribution Crate Global Trigger Processor (GTP) receives all subsystem Level 1 data streams Trigger decisions made in GTP and distributed to all crates via the Trigger Distribution (TD) modules in the Trigger Supervisor Crate 14

Global Trigger Crate VXS Pair Map SubSystem Processors are Payload board format and will communicate with two GTP Note TI mapping is identical to Front-End Crate GTP must also provide SD fanout functions for SSP payload modules 15

GTP Specification Update presented at Online Working Group Meeting 16

Virtex 5 devices available now that have high GigaBit transceiver counts capable of managing the data from up to 8 Sub-System Processors GTP logic will effectively have all trigger data from up to 64 Level 1 crates in one FPGA. 17

Quick view of proposed component layout using Virtex 5 FPGA Routing will be dense for Gigabit links from each SSP ( 4 links * 8 SSP ) GTP must manage the SD functions, but this should be feasible Power and cost are reduced from original proposal 18

Discriminator Status (Not truly trigger system hardware, but very nice new development) 16 Channel dual-threshold discriminator prototypes are in house: -2 units for Hall D -5 units for Hall B Prototype modules will be released in the next week for halls to use. Significantly cheaper than V895: -cost < $2,000 Provides several features not found on V895: -32bit scalers on all channels at both thresholds -Calibrated pulse widths: from 8 to 40ns -Trimmed input offset (<2mV error) -Second 34pin output connector is fully programmable. -Able to perform logic based on all channels at both thresholds Ben Raydo 19

16 Channel VME Discriminator Block Diagram Ben Raydo 20

Schedules, work plans for FY11 - FY13 FY10 design projects are at full resource pace!  FADC250  Production version at end of FY10  F1TDC-V2  SSP  Prototype by end of June 2010  TI-TD  Prototype(s) by end of summer 2010  16 Channel Discriminator/Scaler ( Hall B requirement )  7 ‘production’ units under test now  VXS Crate Specification  At the RFI stage and accounts for each year allocated  GTP  Need new EE full time, but we will have to keep moving forward Baseline Improvement Activities (BIA) review upcoming to review FY11 projects Several of the board projects listed above will be available before FY11 begins FY11 will be an intensive year of significant ‘system’ level testing to assure that these boards are ready for final production quantity orders in FY12 21

Summary (Almost identical to 28-Jan summary) FY10 board design projects are on track and prototypes will be here soon!! As always GREAT WORK on keeping these projects on schedule Work activities exceed EE and E-Designer resources, but we push forward, GTP design activities must be started soon independent of new EE hire status 3 nd annual 12GeV Trigger Workshop will – 8 July 2010 FINAL VXS pair mapping has been established so other groups (Hall A, Hall B) are able to start their custom VXS payload modules. (Super BigBite will use VXS payload modules for GEM tracker) Weekly 12GeV Trigger meeting has produced good discussions and ideas for implementation of system level test programs that will be essential for commissioning the DAQ/Trigger/Readout system in the Hall. ( See Alex’s talk) Looking forward, the FY11 – FY13 schedule appears to be reasonable, (labor $$ included), but detailed work activities will need to be created to assure success. 22

Recent Applications with Level 1 Hardware

TID Test/Commissioning setup: Up to 9 crates Can be supported TD with TS function TI#1 TI#2 TI#8 SD#1 SD#2 SD#8 FADC#1 FADC#16 FADC#2 TID boards Fiber VXS SD#9 Luxury Setup (optional): Subsystem control (two fibers on TI) TSSD TD#1 TD#2 TD#16 TI#1 TI#2 TI#8 SD FADC#1 FADC#16 FADC#2 TID boards VXS Fiber VXS TID#n SubSys TS TID#m SubSys TS

PREx -- HALL A Moller Polarimeter Application SCIN TILL ATOR CAL ORI MET ER LEFT SCIN TILL ATOR CAL ORI MET ER RIGHT FADC ON BOARD SCALER (COUNTER) to be read out by a separate trigger (helicity and gate bits) at the helicity cycle of 30 to 2kHz. CL and CR CL and SL CR and SR CL and CR and SL and SR CL and CR and (SL and SR delayed > 100 ns) TRIGGER condition (or): CL.AND.CR prescaled from 1 to at least 1000 CL prescaled from 1 to at least 1000 CR prescaled from 1 to at least 1000 CL = Σ I=1,4 ∑ j=1,2 P J I > threshold CR = Σ I=1,4 ∑ j=1,2 P J I > threshold SL = (Σ J=1,2 S1 J > threshold) or (ΣJ=1,2 S2 J > threshold) or (ΣJ=1,2 S3 J > threshold) or (ΣJ=1,2 S4 J > threshold) SR = (Σ J=1,2 S5 J > threshold) or (ΣJ=1,2 S6 J > threshold) or (ΣJ=1,2 S7 J > threshold) or (ΣJ=1,2 S8 J > threshold) When TRIGGER condition is met, send data that cause TRIGGER H. Dong

Trigger application example Implemented with CAEN1495 B. Raydo

CPU “davw1” VXS crate CPU “davw5” VXS crate *SSP function embedded inside 2 nd CTP Trigger Decision Trigger Clock Sync Busy L1 Crate Sum 10Gbps Stream 2 Fully Prototyped Front-End Crates 16 test signals distributed to 6 FADC boards

Synchronized Multi-Crate Readout CTP #2 is also acting as an SSP (by summing the local crate + CTP#1 sum over fiber A programmable threshold is set in CTP, which creates a trigger when the global sum (6 FADC boards => 96 channels) is over threshold. Example test with a burst of 3 pulses into 16 channels across 2 crates/6 FADC modules A 2μs global sum window is recorded around the trigger to see how the trigger was formed: Example Raw Event Data for 1 FADC Channel: B. Raydo

Input Signal to 16 FADC250 Channels: Raw Mode Triggered Data (single channel shown only): Global Sum Capture (at “SSP”): Runs at 250kHz in charge mode Latency: 2.3µs(measured) + 660ns(GTP estimate) < 3µs 2 Crate Energy Sum Testing Threshold applied to global sum (96 digitized channels) produces 3 triggers. Raw channel samples extracted from pipeline shown for 1 channel. B. Raydo

Synchronized Multi-Crate Readout Rates FADC event synchronization has been stable for several billion ~150kHz trigger rate. Have run up to 140kHz trigger rate in raw window mode, up to 170kHz in Pulse/Time mode. Ed Jastrzembski has completed the 2eSST VME Interface on FADC allowing ~200MB/s readout B. Raydo Single Crate 12 signals distributed to four FADC250 18% Occupancy

GlueX Level 1 Timing