1. Hall D Level 1 Trigger Dave Doughty 1/10/2008 Hall D Collaboration Meeting

2. Christopher Newport University Outline I.Intro to the Trigger II.Photon Energy (Tagger) III.Track Count Processing IV.Subsystem Processors and Global Trigger Crate V.Global Trigger Processors VI.Trigger Timing VII.Conclusions

3. Christopher Newport University L1 Trigger – Current plans Five separate subsystems –Barrel Calorimeter - compute energy –Forward Calorimeter - compute energy –Forward TOF - compute number of tracks –Start Counter - compute number of tracks (?) –Tagger – compute photon energy (useful at lower lum) Each subsystem computes continuously –Goal - At speed of the FADC pipelines - 250 MHz Global Trigger Processor “combines” all five subsystems –4 level hierarchy: Board -> Crate -> Subsystem -> Global

4. Christopher Newport University Level 1 Trigger Block Diagram BCAL SUM FCAL SUM TOF TRACK COUNT START COUNTER TRACK COUNT TAGGER ENERGY PAIR SPECTROMETER ENERGY SUM PROCESSOR SUM/TIME (8 INPUTS) ENERGY SUM PROCESSOR SUM/TIME (8 INPUTS) FADC -VXS- -Fiber links- 12 Crates -Fiber links- 12 Crates ENERGY SUM PROCESSOR** SUM/TIME (8 INPUTS) ** Process Track Counts -Fiber links- 2 Crates * Longest Link * -Fiber links- 2 Crates -Fiber link- 1 Crates GTP Select FCAL Energy, BCAL Energy, Photon Energy, AND Track Counts F(x) TRIGGER SUPERVISOR ----------------- CLOCK TRIGGER SYNC ROC CONTROL Signal distribution to Front End Crates (Fiber Links)

5. Christopher Newport University The (VXS) Serial Backplane Convert Board Data to serial data –16 bits @250 MHz -> 4 Gbit/s Link speed of 3.125 Gbps => data rate => 2.5 Gbps –Packet overhead also! Multiple “lanes” likely necessary –But two lanes will work fine –With four lanes - redundancy for free! –Higher speed in the future will also help Use VXS “Switch Fabric” –VXS has “familiarity advantage” of VME at JLAB

6. Christopher Newport University VXS Crate

7. Christopher Newport University VXS Serial Routing

8. Christopher Newport University photon beam exits from tagger inside vacuum, continues 70 m down to collimator cave and Hall D 1 mm Alhodoscope exit window is 1 mm Al 1 cm Alother walls of vacuum chamber are 1 cm Al 5 mm Alexposed side of electron exit channel is 5 mm Al Tagger in the Trigger broad-band focal plane hodoscope –144 readout channels –short scint. rods –max rate/chan: 2x10 6 (10 7 ) focal plane microscope –120 readout channels –600 scint. fibers –max rate/chan: 2x10 5 (10 6 )

9. Christopher Newport University Microscope scintillating fiber design focal plane electron trajectory SiPM sensors scintillating fibers clear light fibers Design parameters Design parameters  square scintillating fibers  size 2 mm x 2 mm x 20 mm  clear light guide readout along electron direction  aligned along electron direction for reduced background sensitivity silicon photomultipliers SiPM devices  readout with silicon photomultipliers (SiPM devices)

10. Christopher Newport University Microscope electronics SiPM “bases”  SiPM “bases”  Programmable constant fraction discriminators: 120 channels  High resolution tdc (F1TDC): 120 channels  Pulse height needed:  Pulse height needed: FADC is a good choice: 120 channels  essential for setup  useful for monitoring performance  rates alone give incomplete information  USED TO PRODUCE TRIGGER!  Scalers counting on all channels: 120 channels bias voltage regulation  provides bias voltage regulation:  regulation range: 48 ± 8 V  regulation stability: 0.1 V  regulation control: 0.1 V steps  reasonable packing density: 600 channels on 20x10 3 cm 2

11. Christopher Newport University Fixed array electronics  Conventional 1” phototubes, bases  High voltage for phototubes: 144 channels  Timing needed:  Timing needed: High resolution tdc (F1TDC): 144 channels  for measuring coherent bremsstrahlung spectrum  time is compared with pair spectrometer trigger can run concurrently with experiment  at low intensities (10 7 ) can run concurrently with experiment only for special low-intensity runs  at high intensities (10 8 ) only for special low-intensity runs  Pulse height needed:  Pulse height needed: FADC may not be a good choice: 144 channels (YES IT IS – if you want to ever trigger on it!)  essential for setup  at high intensities, monitor spectrum in integrating mode  Scalers counting on all channels: 144 channels

12. Christopher Newport University Tagger trigger electronics TaggerOR –fast logical OR of 120 channels from the microscope –sent over fiber to GlueX trigger electronics Too primitive! We can do better! –Select channels in microscope –Optionally include main hodoscope –Without any extra hardware!

13. Christopher Newport University Microscope - “Board Tag Word” For microscope, not interested in a “sum,” we want an energy “tag” (or two) Have one crate of FADCs for the microscope –@16 Flash ADC channels/board -> 8 boards Use “hit bit” on each FADC channel as energy tag! –16 bits – every 4 ns! –These 16 bits are exactly like the 16 bit board sum produced by the FADC for the calorimeters, except for the interpretation! For calorimeters - 16 bit “board sum” every 4 ns For microscope – 16 bit “board tag word” every 4 ns Board tag words from all boards sent to “crate tagger processor” –120 bits into crate tagger (at 250 MHz)

14. Christopher Newport University Microscope Photon Energy Tag Use one “Switch Slot” for Photon Crate Trigger Processor –FADCs in payload slots –All “board tag words” sent serially to Photon Crate Energy Tagger –(Note that Photon Crate Trigger Processor is same hardware as Crate Energy Summer – both are now called Crate Trigger Processors) Photon Crate Trigger Processor (CTP) produces a 120 bit composite energy tag word from all microscope channels Can download an energy “mask” to Photon CTP –CTP board is loadable via TI board! Address of two “highest” or “lowest” energy tag bits are sent to Tagger Subsystem Processor –Need 7 bits to encode 120 bits, with 16 bit word can encode 2 tag bits –The “rate conundrum”

15. Christopher Newport University For Hodoscope – Duplicate the System? With 144 channels need 8 bits to encode each energy tag. Two hits still fit into one 16 bit word! No new development! Option to include in trigger (or use by itself)

16. Christopher Newport University Track Processing – at Front End Easy in principle - just add up discriminators –How many layers (are there multiple layers)? –Which disc setting (is there a separate for trigger?) –Programmable time window –At pipeline resolution of 250 MHz - 4 ns time stamp –Only one crate of electronics for TOF? –Use for Start Counter as well? Create crate track count –Use VXS backplane for communication –Can send any data desired over backplane –Track CTP computes total track count in crate Time stamped track count data sent to Track Subsystem Processor (TSP)

17. Christopher Newport University From Crates to Global Processor Must move crate data (energy, track counts, photon energy) to the global trigger processor crate. Use four lanes of fiber optics!

18. Christopher Newport University HFBR-7934 Fiber Module - POP4 compliant Fiber Optic Module. Uses 12 fiber MTP/MPO, 50/125μm multi-mode. - 8 Used Fibers: 4 Tx, 4 Rx @ 3.125Gbps each for a total bandwidth of 12.5Gbps in each direction - Small footprint allows 8 modules to fit VME 6U slot - up to 150m range with 500Mhz-Km fiber, or up to 350m range with 2000Mhz-Km fiber - Interfaces directly with Xilinx/Altera gigabit transceivers on FPGA

19. Christopher Newport University Crate Sum -> Energy Sum Processor

20. Christopher Newport University The Global Trigger Crate Global Trigger Crate has eight SubSystem Processors (SSPs) on one logical “Side” Global Trigger Crate has eight Global Trigger Processors (GTPs) on other logical “Side” SSPs are connected to the GTPs via a custom backplane –“VXS-Like” –8 x 8 Fully Connected Each SSP talks to each GTP via a four-lane Aurora Backplane Link Each SSP sources eight four-lane links to the backplane Each GTP sinks eight four-lane links from the backplane

21. Christopher Newport University The Global Trigger Crate (logical view) SSP Array GTP Array bcal ESPs fcal ESPs tof TSP strt TSP phot ESP VME/ VXS Clk/ Trig In Trig Out

22. Christopher Newport University The Global Trigger Crate (physical view) SSP Array GTP Array bcal ESPs fcal ESPs tof TSP strt TSP phot ESP VME/ VXS Clk/ Trig In Trig Out GTP Array

23. Christopher Newport University Global Crate Backplane Layout In Logical Layout 64 4-lane signals must be routed across center In Physical Layout 32 4-lane signals must be routed across center In “Normal” VXS 18 4-lane signals are routed to each of two center boards Will route!

24. Christopher Newport University A Similar but Smaller Backplane (Bustronic)

25. Christopher Newport University Subsystem Processors (SSPs) Each of the subsystem processors resides in Global Trigger Crate All subsystem processors are same physical PC boards! Each SSP can receive up to eight four-lane “crate data links” Each SSP has multiple Xilinx FPGAs for receiving and processing data from crates Some SSPs use two boards (because of crate count) –If so – both board “Partial Results” sent to global processor Eight SSPs are needed: –Two for BCAL – Energy Subsystem Processor (ESP) –Two for FCAL – Energy Subsystem Processor (ESP) –One for TOF – Track Subsystem Processor (TSP) –One for Start Counter – Track Subsystem Processor (TSP) ???? –One for Tagger – Photon Energy Subsystem Processor –One spare! Each subsystem processor sends time-stamped Subsystem Event Reports (SER) to all Global Trigger Processors

26. Christopher Newport University Global Trigger Processors (GTPs) Run at the same 250 MHz pipeline speed as the subsystems Receive (via backplane) data from all subsystems –TTOF - Tracks Forward TOF –EFCal1 - Energy Forward Calorimeter (partial sum 1) –EFCal2 – Energy Forward Calorimeter (partial sum 2) –EBCal1 – Energy Barrel Calorimeter (Partial sum 1) –EBCal2 – Energy Barrel Calorimeter (Partial sum 2) –Tagger – Photon Energy –STRT – Track count in Start Counter ? Adds partial sums (but can use independently) Adjusts data to match time-stamps Does “global trigger computation” –Z >= TFM*TTOF + EFM*EFCal + RM*((EFCal +1)/(EBCal + 1)) –Or something similar

27. Christopher Newport University GTPs - II Each GTP has multiple internal processors (FPGAs), each looking for one of four triggers With eight GTPs allows 32 different triggers Output to electronics is “Programmable Delay” after computed trigger event time –Produces a result with constant latency (relative to time-stamped data) –Simplifies pipeline “lookback” for front end electronics - it’s fixed Interfaces with TS

28. Christopher Newport University Trigger Timing –Board sum 64 ns –Transfer to crate sum board 512 ns –Crate sum 64 ns –Link to subsystem512 ns –Subsystem trigger processing256 ns –Transfer SER to GTP512 ns –GTP 256 ns –Level 1 output to FEE256 ns TOTAL = 2.432  S - design FEE for 3.5  s

29. Christopher Newport University Conclusion – Where is Level 1? In relatively good shape! –Thanks to electronics group here at JLAB –Have plans and/or proof of concept for most of system –Minimal number of boards to be developed (but of course more “software” versions) Crate Trigger Processor (CTP – simple prototype done) SubSystem Processor (SSP) Global Trigger Processor (GTP) Things we still need –Re-do of L1 simulation Current simulation dates from 2001 Chinese post-doc here at the lab –Track Count Processing

30. Christopher Newport University SSP ->Energy Sum Processors (ESPs) Used for both BCAL and FCAL Logic of both ESPs is essentially same as crate sum Each ESP uses data from all (up to eight) its crates Matches time-stamps and adds data –Knows time-stamp offsets for its subsystem (if any) Passes energy sum results to all trigger processors –SER looks like: –Time resolution is 4 ns

31. Christopher Newport University Where the ESP sits in the Pipeline

32. Christopher Newport University ESP Block Diagram (from Ben)

33. Christopher Newport University SSP ->Track Subsystem Processor (TSP) Used for TOF and perhaps Start Counter Logic of both TSPs is essentially same as crate sum Each TSP uses data from all its crates Matches time-stamps and adds data –Knows time-stamp offsets (if any) Passes total track counts when over threshold to trigger processors SER looks like: –Time resolution is 4 ns

34. Christopher Newport University SSP -> Tagger Energy Processor (TEPs) Process data from Microscope and Hodoscope (?) Matches time-stamps to correlate (if desired) data Pass tagger energy results to global processor (in same crate) –SER looks like: –Time resolution is 4 ns