1. System-On-a-Programmable-Chip (SOPC) Implementation of the Silicon Track Card (STC) Thesis Defense By Arvindh-kumar Lalam Department of Electrical and Computer Engineering Florida A&M University – Florida State University College of Engineering

2. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Outline DZERO Experiment Silicon Track Card (STC) SOPC Implementation and Validation Content Addressable Memory (CAM) Hit-Filter Implementation using a CAM Results and Conclusions

3. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Proton Anti-proton Collision Study the properties of known particles Eg. ‘top’ quark Look for the unknown

4. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering DZERO (D0) Experiment The DZERO Experiment is conducted in Tevatron Collider, at Fermi National Acceleration Laboratory proton & anti-proton are made to collide at high velocities in the TeVatron collider The beams cross every 132 ns The TeVatron Collider

5. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering D0 Detector Fiber Tracker Fiber Tracker (CFT) Identifies trajectory information - “tracks” Silicon Tracker Silicon Tracker (SMT) Contains Silicon charge collectors - “strips”

6. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Particle tracks Cross-section of Fiber Tracker (CFT)

7. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering D0 Trigger L1 Framework L1 CAL L1 CFT L1 Muon L1 FPD CAL FPS/ CPS CFT SMT Muon FPD L2 Cal L2 Ps L2 Muon L2 STT L2 CFT Level 3 L2 Global Trigger Detector Level2Level 1 SMT L2CFT preprocess SMT data find clusters associate clusters with L1CTT tracks fit trajectories L1CFT L3

8. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering The Level_2 STT SMT L2CFT preprocess SMT data find clusters centroids associate clusters with L1CTT tracks (finds hits) fit trajectories L1CFT STC TFC FRC (roads) L3

9. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering D0 Trigger - CFT A Layer CFT H Layer 2 mm road centroids “clusters” : Groups of strips SMT Layers “Si” strips “centroid”: Centroid of a cluster “road” : Track information translated for the STC “hit” : A centroid that falls in a road hits STC

10. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering STC - Functionality Reformats received “strip” data Finds “Clusters” and their “centroids” Identifies “hits” Stores intermediate data for debugging Implements a contention scheme Several STCs function simultaneously Operates at PCI 33 MHz

11. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering STC - Main Data Path Control Lines Main Control Data Lines Handshake Signals Control Lines Hits Downloaded Parameters Control Logic Hit Filter Strip Reader Roads from FRC SMT Data Centroid Finder L3 Buffers To L3

12. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Control Logic and Channels.. Control and Feedback Signals SMT Data (strip information) Control Logic Channel 0 (STC0) Channel 1 (STC1) Channel 7 (STC7)... 7... 1 0 To rest of L2STT Control Logic designed at BU acts as an interface Each Control Logic controls 8 Channels (STCs) STC receives SMT data directly from SMT “commom data bus” is used to download hits

13. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering System-On-a-Programmable-Chip (SOPC) Discrete PCB components? SOPC Altera APEX II EP2A90 7M gates: 1.5Mbits SRAM Xylinx Virtex E XC2V10000 10M gates: 3.4 Mbits SRAM Altera APEX 20KE EP20K600EBC652-1X Accommodates 1 STC

14. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering SOPC - Advantages The circuit can be fit into a single device Occupies smaller area on the board Board-design interconnects are less complex Internal propagation delays are predictable

15. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Computer Aided Design Tools Entry and Functional Simulations: Quartus II, Active HDL 4.2 Entry in VHDL/Schematics Synthesis: Quartus II, Synopsys FPGA Express Simulation and Configuration: Quartus II

16. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering SOPC Implementation of STC Control Logic BU Silicon Track Card FAMU-FSU COE Used Test memory space to store test vectors of SMT data Prototype Testing Board

17. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering SOPC Implementation - Hit download

18. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering SOPC Implementation - Result Contention is successfully resolved ChannelHits TrailersHit-words STC07 18 STC1 212

19. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering STC - Resources Device Family ChipLogic Elements Memory bits Pins FLEX (CPLD) EPF10K100EB C356-1 4,340 (83%)10,532 (21%) 257 (94%) EPF10K200EG C599-1 2,941 (29%)79,424 (80%) 466 (99%) EFF10K200SF C484-1 1,860 (18%)10,692 (10%) 292 (79%) Total10,36196,612 829 APEX (SOPC) EP20K600EBC 652-1X 6,744 (27%)105,828 (33%)262 (53%)

20. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering 1 3 2 Found Encoded Address Data 100 111 001 000 00 01 10 11 DataAddress 4 x 3 CAM with Encoded Output Content Addressable Memory(CAM) 001 1 - 10 - 100 111 001 000 - 010 0 - X -X - 100 111 001 000 100 111 001 000 - 100 1 00 - 011 0 - X - A memory like RAM and FIFO Takes data as input and provides the location Output can be “encoded” or “unencoded” A “found” signal is used to signal presence of data

21. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Address (binary) Data represented in the CAM Equivalent Word decimalbinary 001 0 0 0 1 012, 3 0 0 1 0 0 0 1 1 0 0 1 d 104, 5, 6, 70 1 0 0 0 1 0 1 1 0 0 1 1 1 0 1 d d 110, 4, 8, 120 0 0 1 0 0 1 0 0 0 1 1 0 0 d d 0 0 Don’t-cares can be used to represent multiple digital words A don’t-care (d) represents both ‘1’ and ‘0’ CAMs that accommodate don’t-cares are called Ternary CAMs Eg: APEX CAM Don’t cares

22. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering APEX CAM 1 4 2 Found Encoded Address Data 0001 001d 01dd dd00 00 01 10 11 DataAddress 1100 1 - 11 - 0001 001d 01dd dd00 - 1001 0 - X -X - 0001 001d 01dd dd00 0001 001d 01dd dd00 - 0100 1 10, 11 Memory blocks of Altera’s APEX chip can be used as a Ternary CAM The data can be stored in two ways During power-up (using an.mif file) During run-time

23. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Previous Hit Filter Receives roads and centroids Internally stores roads Uses ‘hit-match’ modules to find if a centroid falls in the roads When a centroid falls in a road, it is a hit Each ‘hit-match’ generates a bit ‘1’ for hit 21…1110... 0 Upper AddressLower Address Road Word

24. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Previous Hit Filter – Block Diagram Comparator Module 46 “hit-match” modules in parallel 11 centroid 46 comparator word Hit - Format (Encoder) 32 hit 22 road word road select 6 Contains 46 ‘hit-match’ modules Each of the centroids is checked in all roads The locations of ‘1’s are encoded to generate a hit-word Hit-format, designed in VHDL, uses Finite State Machine Hit-format module sequentially searches for hits.

25. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Hit Filter – CAM-only model Uses memory blocks instead of a combinational circuit (comparator) Set of all the words existing between the road boundaries is called a “road-set” Each road-set can be minimized to 12 words by using don’t cares “road-sets” of each road are stored in the memory

26. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Actual road-set 1 0 0 0 0 0 0 1 2 0 0 0 0 0 1 0 0 0 0 0 0 1 1 3 0 0 0 0 1 0 0 0 0 0 0 1 1 1 4 0 0 0 1 0 0 0 0 0 0 1 1 1 1 5 0 0 1 0 0 0 0 0 0 1 1 1 1 1 6 0 1 0 0 0 0 0 0 1 1 1 1 1 1 7 1 0 0 0 0 0 0 1 0 1 1 1 1 1 8 1 1 0 0 0 0 0 1 1 0 1 1 1 1 9 1 1 1 0 0 0 0 1 1 1 0 1 1 1 10 1 1 1 1 0 0 0 1 1 1 1 0 1 1 11 1 1 1 1 1 0 0 1 1 1 1 1 0 1 12 1 1 1 1 1 1 1 Minimized Road-Set 1000 0000001. 1000 1111110 Minimized road-set 0 0 0 0 0 0 1 0 0 0 0 0 1 d 0 0 0 0 1 d d 0 0 0 1 d d d 0 0 1 d d d d 0 1 d d d d d 1 0 d d d d d 1 1 0 d d d d 1 1 1 0 d d d 1 1 1 1 0 d d 1 1 1 1 1 0 d 1 1 1 1 1 1 0

27. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering CAM-only model – Implementation Road-set Generator 11 centroid 11 Road-set word CAM found 22 road Control signals 10 location Hit create 32 hit

28. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering CAM-only Model - Functionality Storing roads The road-set is minimized by using the “don’t cares” The minimized road-set is stored in an APEX CAM The CAM needs 50 clock cycles to store each road-set Checking for hits Each of the centroids is given as input to the CAM If the centroid is found in the road-set, CAM returns all the encoded locations. CAM takes only two clock cycles to find the location of first hit

29. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Hit Filter – With New Encoder Uses previous comparator block and a new “hit-word generator” block The locations of ‘1’s in the comparator word are encoded using a CAM

30. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Hit Filter – Implementation Comparator Module 46 “hit-match” modules in parallel 11 centroid 46 comparator word Hit -Word Generator (Encoder) 32 hit 22 road word road select 6

31. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering CAM as Encoder 1 4 2 Found Encoded Address Data d d d 1 d d 1 d d 1 d d 1 d d d 00 01 10 11 DataAddress 0001 1 - 00 - d d d 1 d d 1 d d 1 d d 1 d d d - 0000 0 - X -X - d d d 1 d d 1 d d 1 d d 1 d d d d d d 1 d d 1 d d 1 d d 1 d d d - 1001 1 00, 11 ddd1 3 dd1d 2 d1dd 1 1ddd 0 0123 4 x 4 Encoder Map

32. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Hit Filter Encoder Map 15 x 15 Encoder Map 31 x 31 Encoder Map 46 x 46 Encoder Map

33. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Hit-Word Generator 5 6 46 Comparator Word HIT GENERATOR CAM 31x31 CAM 15x15 31 15 32 HIT Control Signals

34. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Hit Filter Results 6 roads (consecutive) 6 roads (distributed) 46 roads Sequential search (contains comparator) 646 CAM only 270 *310 *2070* With CAM block in hit- word generator (contains comparator) 646 * This depends on the upper and lower words of the road. The quoted figures correspond to the worst possible case. Number of clock cycles required for storing road information

35. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Hit Filter Results 6 roads (consecutive) 6 roads (distributed) 46 roads Sequential search (contains comparator) 32150232 CAM only 10 50 With CAM block in hit- word generator (contains comparator) 10 50 Number of clock cycles required for finding hits

36. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering STC Results STC6 consecutive roads 46 roads6 distributed roads Event1Event2Event1Event2Event1Event2 Previous 4.878  s 15.0  s 16.48  s 76.03  s 11.636  s 51.78  s Upgraded 4.03  s 6.909  s 5.242  s 19.06  s 4.03  s 6.909  s % decrease in time taken 17%54%68%75%65%87% Event 1 : SMT data for a simple event Event 2 : SMT data for a complex event

37. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Conclusions SOPC implementation was successfully verified The upgraded STC shows an improvement of upto 87%

38. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Future Work The number of roads Hit-Filter can accommodate can be increased

39. FAMU-FSU College of Engineering Department of Electrical and Computer Engineering Acknowledgements National Science Foundation and the US Department of Energy. Boston University Faculty: Heintz, Narain, Popkov Engineers: Earle, Hazen Students: Kevin, Zabi Florida State University – Physics Faculty: Adams, Prosper, Wahl Postdocs: Tentindo-Repond Florida A&M University – Florida State University COE Faculty: Perry Students: Lolage, Meyers, Roper, Saunders Altera, Aldec, Synopsys