1. Possibilities for CPM firmware upgrade
Juraj Bracinik, Richard J. Staley
(University of Birmingham)
L1Calo upgrade meeting, QMUL 3/2/2011
Introduction
CPM Hardware Limits
Possible data transfer schemes
near future plans
Medium term plans
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

2. Motivation for CPM upgrade
Atlas trigger upgrade
A long process
Several stages (Phase 1, Phase 2, Phase 1.5 …)
Time schedule (slowly) being defined, changing from time to time
First part of upgrade
Existing granularity of calorimeter signals
the same definition of “trigger objects” (electrons, taus, jets, Et, Emiss) as in pre-upgrade L1Calo
No change to CPM algorithms
Main performance gain from adding topology
Keep most of existing L1Calo HW
Add CMM++
Or (hopefully) CMM++ + TP
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

3. UK L1Calo upgrade meeting 3/2/2011
I. CPMs in pre-upgrade L1Calo
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

4. CPM architecture – recapitulation I
Most of work done in three types of FPGAs:
20 serializers- processing of input data streams, data distribution on-board and across backplane
8 CP chips – look for e/tau objects (RoIs)
8 EM and 8 EM/TAU thresholds
2 hit mergers – calculate total number of objects found by CP chips
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

5. CPM architecture – recapitulation II
CP chips:
Calculate energies in cluster, isolation and had veto energies
Compare with pre-loaded thresholds
There can be up to 2 objects (RoIs) on each chip (in left/right half of the chip)
Information about these objects (which thresholds were passed) are sent on 2x16 data lines to 2 Hit mergers
One Hit merger looks at EM thresholds, the other one at TAU/EM
Hit mergers:
Calculate total number of objects passing each threshold (saturate at 7)
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

6. II. limits of existing CPM hardware
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

7. Factors influencing CPM upgrade
Speed of backplane transfer from CPM(s) to CMM(s)
running at 40 MHz
Bottle-neck in pre-upgrade L1Calo
Transfer speed between CP chips and hit
merger chips
Running at 40 MHz
Size of CP and Hit FPGAs
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

8. UK L1Calo upgrade meeting 3/2/2011
Backplane transfer I
Tried to increase transfer frequency to 160 MHz
Tested with pseudo-random patterns and by test patterns from serializer playback
Built simple test board that allows to probe backplane signals in CMM slot
With termination on both sides (CMM and CPM) the signal quality is very good!
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

9. Backplane transfer II Possible to run backplane at 160 MHz!
Termination on CMM side improves signal quality, but increases dissipated power in CMM slot – can be a problem
Tested signals with termination on source only
Still very good
Should work with termination on CPM only!
Possible to run backplane at 160 MHz!
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

10. Transfer from CP chips to Hit mergers (on board) I
Data lines between CP chips and Hit Mergers run currently at 40 MHz
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

11. Transfer from CP chips to Hit mergers (on board) II
Different length of lines on the board limit possible rate
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

12. Transfer from CP chips to Hit mergers III
Tested with dedicated firmware for CP chip and Hit merger
CP chip sending parity protected data at 80 MHz
Hit merger checking parity of received data
Scan of parity errors as a function of phase difference between Deskew1 clock (used by Hit merger) and Deskew2 clock (used by CP chip)
Parity error zone width dominated by spread of individual chips
Possible to find error-free zone of reasonable width
Possible to run these lines at 80 MHz!
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

13. CP and HIT FPGAs – available resources
CP chip is rather full, but it is possible to add more functionality in Hit Merger (smaller FPGA)!
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

14. III. Possible CPM to CMM++ transfer schemes
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

15. Summary of available bandwidth
Backplane at 160 MHz – 4x current data rate
Each Hit FPGA to CMM => 96 bits/BC
CP chips to Hit mergers at 80 MHz – 2x current rate
Each CP chip to 1 HM => 32 bits/BC
Some additional processig/routing possible in Hit FPGA
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

16. How to use available bandwidth?
Backplane can have 25 lines run at 160 MHz
Some information about RoIs that were found
Parity bit
Clock forwarding
Possible to use one line for both parity bit and clock forwarding (line 24, used for parity bit by CMM)
Rising edge gives clock phase information
Parity bit corresponding to data in 4 160MHz ticks encoded in position of falling edge
Leaves 24 lines for data (lines 0-23)
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

17. What to send – 10 RoI scheme
First send map of locations where RoI's were found (16 bits, because there can be up to 2 RoIs in each of 8 CP chips)
Then send hit bits (which trehslods were passed) for first 10 RoIs (8 bits/RoI)
Remaining RoI simply discard (simulation shows it is very rare)
Many advantages:
Simple (involves only change to Hit Merger)
Can do a lot of nice things (topology, overlap removal)
Limited information about each RoI (8 bits)
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

18. 10 RoI scheme – latency and resource usage
Richard has implemented 10 RoI scheme in FW
Checked with serializer playback feeding CP chips
Looking at signals with scope
Looks fine...
Latency:
Gate-level simulations :38 ns (1.5 Bcs)
Standard hit-merging design: 30 ns
Some latency penalty, but not big
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

19. 10 RoI scheme – latency and resource usage
Use of resources:
Design fits into the FPGA
Uses major fraction of resources
This is about how complex we can go in Hit FPGAs
A bit of resources still available..
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

20. 6+2 scheme (6RoIs and 2 highest ET ones)
Simulations indicate most events have maximum of ~6 RoIs/CPM
10 RoI scheme doesn't allow for detailed energy and position information to be sent
“Improved” 10 RoI scheme (6+2 RoI scheme):
Send 16 bit RoI location map
and threshold bits for 6 RoIs (8bits each)
And detailed information about 2 highest ET RoIs:
Transverse energy (ET) – 8 bits
Detailed location – 5 bits
Isolation – 3 bits
EM information into one CMM++, HAD into the other one
Our prefered scheme (for the moment):
it gives us some backward compatibility
allows for more energy thresholds (low ET thresholds in CPM, high in CMM++ or TP
Use topology further downstream
Seems to be possible, work (latency, resources …) in progress...
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

21. UK L1Calo upgrade meeting 3/2/2011
Energies only scheme
In this scheme (almost) no thresholding in CPM
All localized energy deposits (clusters) passing (low) threshold are transferred to CMM++
One CMM receives:
16 bin RoI map
Detailed information about 6 RoIs:
ET 8 bits
Fine location 2 bits
Isolation – 3 bits
One CMM can receive EM information, the other one HAD
All thresholding and topological processing done in CMM++ and Topological Processor
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

22. How to pick the transfer scheme ?
Not sure if any of these schemes makes much sense :-(
A lot of interesting studies needed:
How many RoI do we really need
Is energy sorting in Hit FPGA important (reasonable) thing to do?
If yes, is one (two) highest ET deposits in CPM enough
Do we need fine RoI location?
All 8 energy bits?
Would be nice to have a list of benchmark channels we would like to trigger on with these schemes...
Are we looking for one scheme or rather thinking about gradual upgrade in several steps?
Maybe start with 10 RoI scheme
Next step 6+2 RoI
Then energy only
Maybe even start with existing CMMs and 80MHz backplane?
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

23. Future plans – CPM FW upgrade
Continue playing with CPM FW, checking possibilities
Several things haven't been touched yet:
Proper testing of existing pieces of FW
Readout
Details of CP chip FW
Need to have a look at how to send energies out
Plan to look at these things with priority depending on CMM++ time scale and amount of new exciting ideas (either from simulation or from data-taking)
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

24. Future plans – middle term I
Traditional areas of interest at Bham:
High-speed PCB design
Optical links
FW development
Online SW
Physics studies and simulation
It is not clear what type of HW we (UK L1Calo) are going to build and when
Maybe we will know this afternoon :-)
We know it won't be VME, probably ATCA
A lot of new technologies and tools to learn
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

25. Future plans – middle term II
Started discussion about building an ATCA based HW demonstrator:
complementary to GOLD, possibly learning from GOLD experience
Look at various options for board control:
One board connected via Ethernet, serving as master, other boards in crate connected via PCIe
Each module having Ethernet access
Controlled by a dedicated processor (PIC, ARM) on board
Controlled by hard/soft core in one of big FPGAs
Running embedded code or full (Linux) OS?
Some big, fast and expensive FPGAs!
Some optical links!
Designed at Bham or in collaboration with other L1Calo institute(s)
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

26. Future plans – middle term III
Possible shopping list for this year:
Buy ATCA crate
Which type? Would be nice to use the same technology within L1Calo
Buy some test board(s), get experience with modern FPGAs and microcontrollers
experiment/research with module control schemes, Ethernet and PCIe
Try to build an ATCA based demonstrator towards the end of 2011
Exact design depending on the experience with GOLD and L1Calo UK plans
Feedback welcome !!!
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

27. UK L1Calo upgrade meeting 3/2/2011
Backup slides
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

28. UK L1Calo upgrade meeting 3/2/2011
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

29. UK L1Calo upgrade meeting 3/2/2011
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011

30. UK L1Calo upgrade meeting 3/2/2011
J. Bracinik, R.J.Staley
UK L1Calo upgrade meeting 3/2/2011