1. A new concept to use 3D vertical integration technology for fast pattern recognition Ted Liu, Jim Hoff, Grzegorz Deptuch, Ray Yarema Fermilab Questions or Comments: jimhoff@fnal.gov

2. Introduction and Outline The development of 3D technology for the solution of the fast pattern recognition problem is part of a broader, ongoing R&D effort that includes both 2D and 3D solutions. This talk will cover: ◦ An introduction to the problem ◦ A description of the Associative Memory solution ◦ A new concept – VIPRAM – that uses emerging 3D technology

3. 10 33 10 35 10 32 cm -2 s -1 10 34 simulation The Obvious Problem… There are enormous challenges in implementing pattern recognition for a tracking trigger at LHC (L1&L2), due to 1.The much higher occupancy and event rates at the LHC 2.The much more massive detectors 3.The larger number of channels in their tracking volumes There is a clear need to develop/improve the hardware-based pattern recognition technology to advance the state-of-the-art for the future

4. The Challenges To increase the patterns density by 3 orders of magnitude (from the original AMchips) and increase the speed by more than a factor of 3 while reducing power consumption (or at least dramatically reducing the rate of increase of power consumption) 1. [1] Based on the extensive simulation studies by Atlas FTK Collaboration

5. Some Obvious Questions… Can’t we just use what we currently have and just make bigger PC boards or more of them? ◦ No. This results in severe speed bottlenecks and power issues. Can’t we just use commercial CAMs? ◦ No. CAMs are part of the fast pattern recognition process, but not all of it. Alone, CAMs lack certain necessary features, making them unsuitable for fast track triggering.

6. It’s not a CAM; it’s a PRAM A CAM (Content Addressable Memory) is a classical digital system building block One pattern at a time Each CAM cell responds or does not respond to the current pattern There is no memory of previous matches Pattern 1 Match Pattern 3 Match Pattern 7 Match

7. It’s not a CAM; it’s a PRAM A PRAM on the other hand is a Pattern Recognition Associative Memory. Layer 1 Address 4 Match Layer 1 Address 4 Match Layer 3 Address 7 Match Layer 3 Address 7 Match Layer 3 Address 9 Match Layer 3 Address 9 Match Layer 2 Address 1 Match Layer 2 Address 1 Match Layer 4 Address 4 Match Layer 4 Address 4 Match Layer 2 Address 4 Match Layer 2 Address 4 Match Road!

8. History and the “traditional” effort The AMchips were invented and developed in Italy resulting in the AMchip03 which is currently being used by CDF. There is an ongoing effort, led by Italians, to improve on the AMchip03 design. We are now a part of this collaboration. The idea, of course, is to increase pattern density and speed and to optimize the performance. Design in deep sub-micron processes. The current target is 65nm.

9. Limitations in 2D…

10. A Single PRAM Cell (in 2 dimensions) CAM Cells Match Storage Match lines Glue Logic In the older version of the AMchip, the match lines were a source of speed limitation because of their length and capacitance. The Glue Logic was large and slow. Length -> Capacitance -> Reduced Speed

11. THE CONCEPT – VIPRAM Vertically Integrated Pattern Recognition Associative Memory A Reduced Footprint and therefore greater pattern density. Shorter Match lines and therefore greater speed. Less Capacitance and therefore reduced power consumption Each detector layer corresponds to a single tier All communication from “CAM Tiers” to the single “Control Tier” The PRAM concept is tailor- made for 3D design. Much Shorter

12. Another Single CAM Cell (this time in 3 dimensions) Viewing this structure as a pseudo-layout some of the aforementioned benefits become even more obvious. The 3-dimensional design of the VIPRAM makes the PRAM appear like a 2- dimensional array of “tubes”, each dedicated to a single pattern. Communication with the outside world during normal operation is done solely through the Control Tier (the blue tier on top).

13. Pattern recognition for tracking is naturally a task in 3D track road

14. Majority Logic – Old Version Match Lines User-defined Threshold Road Flag Adder Digital Comparator

15. Majority Logic – New Version 0 1 Sel 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 Match1Match2Match3Match4 Match Pattern Pass Transistor Logic

16. Majority Logic – New Approach Majority PatternMeaning 111Perfect Match 0111 Missing Layer 0012 Missing Layers 0003 or More Missing Layers Stage Input Stage Output: Match Stage Output: Mismatch 111 011 001 000 For each stage… In the end…

17. Can 3D exploit even more advantages from the new Majority Logic? Yes. We have divided the 3D design by detector layer (i.e. each CAM Tier is dedicated to one detector layer) Therefore, any logical division by detector layer results in functions that can be sub- divided by tier.

18. Can 3D exploit even more advantages from the new Majority Logic? 0 1 Sel 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 Match1Match2Match3Match4 Match Pattern

19. Readout The top tier (a.k.a. the Control Tier) is a two dimensional array of elements whose position is indicative of its address and that contains an indication of whether or not a road was found. Compare this with a pixel array which is a two dimensional array of elements whose position is indicative of its address and that contains an indication of whether or not a hit was found. In other words, high-speed readout architectures for pixel arrays can and should be used for VIPRAM readout.

20. Design for Simplicity The VIPRAM has two types of tiers, CAM and Control. In the final design, there will be several CAM tiers and only one Control tier. Each CAM tier is functionally identical to the others, but must maintain a unique relationship to the Control tier in order to work. In other words, patterns that come into the Control Tier from Detector “1” must be sent to the CAM tier dedicated to Detector “I”. Similarly, when data is sent from CAM tier #3, the Control Tier must know it came from CAM tier #3 and not some other CAM tier. How can this be done without requiring unique mask sets for each CAM tier?

21. Great minds think alike? Having gone part-way through this design procedure, the collaboration had the opportunity to meet with Bob Patti of Tezzaron who has been involved in 3D memory design from the beginning. Tezzaron’s 3D Memories follow exactly this arrangement of Control Tier and (in Tezzaron’s case) Memory Tier. In other words, we are following a beaten path, not blazing a new trail.

22. The Diagonal Via The Diagonal Via was patented by Bob Patti and Tezzaron in 2000. It converts vertical position to horizontal position and allows a common mask set to provide unique access to each layer.

23. Conclusions and Future Work The VIPRAM is a new concept and now we are developing a collaboration with Fermilab, University of Chicago, INFN and Argonne. ◦ The immediate goal is a proof of principal ◦ The ultimate goal is a 3 order-of-magnitude increase in performance (density+speed). At present, we are seeking funding for the VIPRAM development. You will hear from us again at the next TIPP (please pick a nice place for my wife…)

24. Background

25. Figure 13 - Pass Transistor Multiplexors in the Majority Logic

26. VIPRAM – A Vertically Integrated PRAM Modern technology provides us with another approach…and another dimension. At first, the idea was extremely simple – increase the pattern density by stacking otherwise normal AMchips. The outputs of existing AMchips are already in a daisy chain. The stacked AMchips would not need to “know” that they were part of a stack.

27. VIPRAM – A Vertically Integrated PRAM This was necessarily modified to include “wrapping” an AMchip in circuitry that dealt with the 3D stacking, leaving an AMchip core that was identical to the 2D AMchips that are under development.

29. Not the first to consider 3D Content Addressable Memory Oh and Franzone 1 first suggested the advantages of 3D design on CAMs in 2007 Their idea involved vertically integrating the CAM cell itself so that the Matchline was vertical. This minimized its length and therefore its capacitance. The method is highly impractical since it requires f(N) 3D layers where N is the number of bits in the CAM cell. CAM Bit Cell CAM Bit Cell CAM Bit Cell CAM Bit Cell Matchline 3D Layer 1 3D Layer 2 3D Layer 3 3D Layer N [1] E.C. Oh and P.D. Franzon, “Design Considerations and Benefits of Three-Dimensional Ternary Content Addressable Memory”, IEEE Custom Integrated Circuits Conference, 2007, p. 591

30. Again, this is a PRAM not a CAM There is a perfectly natural, 3D functional division in a PRAM. Each detector layer gets its own 3D layer. The vertical interconnect is not the CAM match line, but the Road line. Moreover, each detector layer has independent data lines for both pattern matching and pattern loading, and this is a natural consequence of this architecture. Match Road! 3D Layer 1 3D Layer 2 3D Layer 3 3D Layer N

31. How can we improve on this design? 4 blocks of 1280 6-layer patterns ~80% AM bank ~20% control & interface Move to another tier in 3D

32. How can we fundamentally improve on this design?

34. Majority block still in standard cell ~ 30% can be also moved to the control/interface tier in 3D within each pattern block

35. Fischer Tree (Mephisto Logic) P. Fischer introduced the Mephisto readout architecture [1]. We found “Fischer Tree” easier to say. It is a self-selecting, self-addressing priority encoding architecture that performs the task in log[N] time. [1]“First implementation of the MEPHISTO binary readout architecture for strip detectors” Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment Volume 461, Issues 1-3, 1 April 2001, Pages 499-504 8th Pisa Meeting on Advanced Detectors

36. Fischer Tree (Mephisto Logic) Fischer Trees can be stacked if need be, so the two dimensional array in the Control Tier can be handled this way. An alternate approach could take each output and push it into a stack. Fischer Tree Col 1Col 2Col 3Col N Fischer Tree …