New Base DAQ Firmware for FONT5/FONT5A Glenn Christian 30/08/12

Motivation Long (and growing) laundry list of preferable additions/corrections to existing FB FW FBFW always teetering on the edge of 2.8ns timing spec –small changes could (randomly) make it fail or meet timing –Most changes not implemented, only if absolutely essential effort made to implement and meet timing (eg internal RC -> MPPR) For CTF3 want to reuse as much as possible of the sampling/DAQ logic, but many changes necessary e.g. different number of samples to be recorded per pulse, no ring clock ala ATF …

Motivation 2 Current FB FW –ATF parameters hard-wired in : ring clock to synchronise the bunches to the trigger, 164 samples (= 1 ring clock cycle - 1 sample) per pulse recorded from all 9 channels –Functions related to FB mixed in with those required for sampling/DAQ Want to –Separate the application level code eg (ATF FB, CTF FF) from the underlying FW for sampling (base FW) –Work with variable fast clock speed, variable number of samples recorded (and channels), with/without ring clock (ext/int) –Base FW not affect the FB latency therefore can Optimise the base to meet timing comfortably, with relaxed tool settings (default settings area/timing optimisation) –Much faster P&R durations for base FW –Give a stable base to build application on –Allow implemention of all features on laundry list etc…

Ultimate goal (but whether it happens or not …) General purpose scope-style DAQ base application –System clock : 200 – 400 MHz (set by ADCs) –Option to use secondary trigger counter (eg RC) – EXT or INT –Variable record length and channels active –Trigger source: EXT, chs 1-9 & level (for trigger on data) – NB: might be useful at CTF –Run/Stop/Single-shot mode Application layer FW or DAQ software can lock down unused functions/variables

Summary of changes (so far) Simulate changes at module level as much as possible Adhere to FPGA design guidelines and verilog-2001 rules as much as possible (Timing) –Verilog keywords as signal names –Avoiding numbering in signal names for duplicate registers (routed as bus by router) –Undefined states in FSMs, tidied up FSM implementations –Global/Async Resets Current FW most FFs have async reset even though most resets unnecessary and the global reset was already synced to the 40 MHz Async resets use LUT resources in the CLBs – stop the design from working at full speed (eg 550 MHz Virtex 5) Not needed anyhow as GSR can be relied upon, provided have access to the JTAG config NO global resets on the fast (eg 357 MHz) clock domain except specific primitives (eg DDR registers) with async reset pin Reset remains on 40 MHz domain but implemented as synchronous not async Other changes to DCM resets and reset sequencing etc

Summary of changes (so far) 2 Alignment monitors –Lots of cross clock domains –Current module specifically assumes 357 MHz system clock (relies on 9 periods of 40 Mhz in one 357) Introduced handshaking signals Reduced the number of iterations in the averaging from 127 to 32 so that align-monitoring will still complete within ~260 us ADC on-time at 200 MHz, cf 357 MHz Needs to be tested in lab –Is 32 iterations enough? –Did they ever work properly anyway/ were they ever tested?

Summary of changes (so far) 3 Timing Synchronisation module –FB version runs all timing related signals off one large counter, including FB specific signals (bunch strobes, amplifier triggers etc) –Removed all signals specific to application and re- coded as an FSM STATE = WAIT -> WARM_UP -> SAMPLE -> ALIGN; Only care about bunch timing wrt trigger, eg align monitor duration tied to 40 MHz –Previously for ATF 12’ counter gave 2 ms range – 1 ms trigger to bunch, 10 us warm up, 1 us sampling, 250 us align_mon Interface to timing signals eg bunch strobe etc via the STATE variable which is brought out

Current Status Above changes allowed timing bottlenecks to be clearly identified and fixed –Main problem in align_monitors (already suspected) : large fanout to DDR registers Register duplication cf pipelining (ie parallising pipelining rather than serialising) brought timing down to 2.71 ns (best ever seen) –P&R times now few minutes cf ~10-20 mins Next worst nets at endpoints of Control register fanout – not looked at for now. –Modules simulate okay, but need to be checked in lab before proceeding (several new warning messages – lost track of!)

Problems with testing So far, have only made changes that should still work with current labview DAQ –Eg by locking variables to ATF values in FW, can check that FW still works as it would at ATF Will need to also make significant changes to the DAQ –Problem with debugging hardware and software simultaneously! –With greater timing slack can probably insert CSP cores to test basic functionality but not full operation of UART (NB can’t run CSP in ISE10 – will need to migrate to newer version but I suspect timing issues in later versions now resolved) –Also suspect I will have to re-learn Labview to be able to efficiently debug system –Mikey python scripts? Also, issue with testing board 1 at 200 MHz (357 MHz filter), board 2 programmable?

Next steps Re-implement DAQ RAMs UART Control registers –Many new control registers needed –Currently two sets of CRs are synced onto appropriate clock domain (357, 40 MHz) Running out of space on 357 domain Actually all signals from UART already synchronous on 40 MHz domain – plan to leave all on that and synchronise 357 registers locally where used. Would like to keep current mapping of addresses for compatability. Will require changes to DAQ software

Other things to consider ISE versioning?? Testing at 200 MHz –Need to check integrity of ADC clocks at this speed Actually two 40 MHz clock domains in logic [1 not through DCM (always runs, drives uart rx, reset), other from DCM0 output (stopped when ADCs running, drives uart tx)] –Worry about synchronisation between the two?