1. FPGA Design Methodology for the CMS Calorimeter Trigger Upgrade Presenting: William Plishker George Zaki, Soujanya Kedilaya, Nimish Sane, Shuvra S. Bhattacharyya University of Maryland February 19, 2009 In collaboration with Wesley Smith, Sridhara Dasu, Michail Bachtis, Kevin Flood, Tom Gorski, David Hinkemeyer, Michael Schulte, Katherine Compton, Tony Gregerson, Ben Buchli, and Amin Farmahini-Farahani

2. 2/16 Outline Goal: To strengthen the connection between developing algorithms and producing implementations on FPGAs and other embedded systems. Motivation & Problem Statement Dataflow Interchange Format Unit Testing Framework Preliminary Design Example –Particle Cluster Finder Future Work

3. 3/16 Introduction A Calorimeter Trigger is a complex system in which we would like to –Update algorithms –Arrive at correct implementations quickly –Achieve high performance But algorithms are far removed from their final implementation (the Implementation Gap) –Low level programming environments –Diverse and changing platforms –Non-uniform functional verification Implementation Gap Abstract representation of an algorithm Low level, high performance, implementation

4. 4/16 Our Solution: DIF with Unit Testing Describe application using the Dataflow Interchange Format (DIF) –Founded in formal models –Platform independent –Ideal starting pointing for analysis Utilize DIF package –Perform analysis and/or synthesis Unit test modules –Uniform framework for testing modules Abstract representation of an algorithm Low level, high performance, implementation DIF description of algorithm DIF description of algorithm DIF Package Automated cross- platform unit testing

5. 5/16 DIF bridges the implementation gap Abstract, formal application description Abstract, formal application description High-performance implementation Dataflow Interchange Format (DIF) Other benefits to beginning with a formal description: –Bounded memory and deadlock detection –Buffer and communication minimization: –Parallel, Multirate loop, or Quasi-static scheduling –Heterogeneous task mapping and co-synthesis –Probabilistic design, Data partitioning, Vectorization, Dataflow Models DSP Designs The DIF Package (TDP) The DIF Package (TDP) DSP Libraries DSP Libraries Dataflow- based DSP Design Tools Dataflow- based DSP Design Tools Embedded Processing Platforms Embedded Processing Platforms The DIF Language (TDL) DIF Specification Signal Proc Image/Video Comm Sys Meta-Modeling PDFBLDF Dynamic CFDFBDF DIF-to-C Algorithms Front-end DIF RepresentationAIF / Porting Static SDFMDSDF HSDFCSDF C DSP Other Embedded Platforms Other Tools Other Ex/Im VSIPL TI Other Autocoding Toolset Ptolemy II DIF-A T Ex/ImPtolemy Ex/Im Java Java VM Ada VDM DIF captures course grain dataflow applications formally To formally describe applications the DIF Language (TDL) is –Designed to capture a variety of dataflow models –Can be used in conjunction with functionally simulatable actor descriptions To facilitate design, the DIF Package (TDP) provides: –Scheduler, simulator, analyzers

6. 6/16 Golden Model Design with DIF Even if a full synthesis flow is not available for a given platform, DIF can be used as a “Golden Model” –DIF enables clear, unambiguous description of an application and the modules in it –Functional simulation provides definitive outputs for comparison with implementations –The DIF description is platform and programming language independent Golden Model design flow –Construct DIF reference model –In parallel or in series construct platform specific implementation –Share testbench and compare results Note: Modules may already be implemented DIF Reference Model DIF Reference Model Platform specific Implemen- tation Sample Datasets Comparator Error Report Golden Model Design

7. 7/16 Example Design with Clustering Algorithm Source: http://www.hep.wisc.edu/~bachtis/public-files/slhc/Algorithm.ppt ClusterThreshold Actor ClusterComputation Actor

8. 8/16 Transform into a Formal Description Cluster Threshold Actor Cluster Threshold Actor Cluster Computation Actor Cluster Computation Actor ECAL 1-4, HCAL 1-4, FG 1-4 (In all, 12 input ports) If Pattern Match ECAL 1-4, HCAL 1-4, FG 1-4 are transmitted Else no tokens transmitted. 38 bit output 11 1 1 1 1 1 1 1 [0,1]

9. 9/16 Write Modules and Unit Tests To ensure modular, functional actors, write functional DIF actors and unit tests –A unit test is “A piece of code written by a developer that exercises a very small, specific area of functionality of the code being tested.” Pragmatic Unit Testing, Andrew hunt and David Thomas Unit tests –Decrease overall debugging time by catching bugs earlier –Can be used on many different types of programming frameworks –Catch special type of bugs ( exceptions, buffer overflow, special inputs, etc…) –Make programmer productivity more consistent

10. 10/16 ClusterComputation Actor Actor fires if all input tokens available Computes ∑ECAL, ∑HCAL, Bitwise OR of FineGrain Bits, Channel Etotal Returns 38 bit output ECAL 1-4 38 bit Output HCAL 1-4 FG 1-4 Cluster Computation Actor

11. 11/16 38b output ECAL 1-4 FG 1-4 HCAL 1-4 For DIF actors a test bench is automatically constructed File reader 1 File reader 2 File reader 3 File writer 1 Text files (sample input provided by user) Expected 38b output Comparator from larger test framework Comparator from larger test framework Cluster Computation Actor

12. 12/16 Project testing framework A project consists of many actors that need to be tested for all input patterns. By dividing the actors in a hierarchical format, all the unit test can be run systematically from a given point in the testing tree. A log file will show the passed and failed test.

13. 13/16 Unit Tests Example: Cluster Threshold test_1 test_2 test_3 Test_Cluster_ Computation expected_output.txt makeme.sh runme.sh Required files optional files expected_output.txt makeme.sh runme.sh expected_output.txt makeme.sh runme.sh inputs.txt Makefile Each test occupies a different subdirectory Contains files to build and run the test –These scripts can be customized for a variety of design environments Expected output is supplied by the user The unit tests can then be run by an automated test suite

14. 14/16 Project Testing Testing HEP Project_1 test_Cluster Test_Overlap_Fileter test_1 test_2 test_3 input.txt output.txt runme.sh makeme.sh Test_Cluster_ Computation Test_Cluster_ Computation Test_Cluster_ Threshold Units themselves may be grouped by the hierarchy present in the application The test framework can run either: –Locally for tests while developing –Globally for nightly builds input.txt output.txt runme.sh makeme.sh input.txt output.txt runme.sh makeme.sh

15. 15/16 Summary & Future Work Our solution to the implementation gap –Describe application functionality in a formal environment like DIF –Use this description as a synthesis starting point or a Golden Model –Develop unit tests with each module to be shared with the DIF and platform specific implementations We will continue working with U. of Wisconsin researchers to advance new tools and techniques to design, test, and upgrade the CMS firmware –More actor implementations in DIF with unit tests (Particle Isolation/ID, Jet Recon, Particle Sorter, and MET,HT,MHT Calc) –Software synthesis path to a hardware description language –Techniques for answering high level design questions

16. 16/16 Questions? http://www.ece.umd.edu/DSPCAD/dif/ Selected Publications –W. Plishker, N. Sane, M. Kiemb, K. Anand, and S. S. Bhattacharyya. Functional DIF for rapid prototyping. In Proceedings of the International Symposium on Rapid System Prototyping, pages 17-23, Monterey, California, June 2008. –W. Plishker, N. Sane, M. Kiemb, and S. S. Bhattacharyya. Heterogeneous design in functional DIF. In Proceedings of the International Workshop on Systems, Architectures, Modeling, and Simulation, pages 157-166, Samos, Greece, July 2008. –C. Hsu, M. Ko, and S. S. Bhattacharyya. Software synthesis from the dataflow interchange format. In Proceedings of the International Workshop on Software and Compilers for Embedded Systems, pages 37-49, Dallas, Texas, September 2005.

17. 17/16 ClusterThreshold Actor Cluster Threshold Actor ECAL 1-4 HCAL 1-4 FGout 1-4 Actor enabled when all Input tokens available INVOKE stage – Compares ECAL and HCAL values to Threshold values Threshold Values are Actor attributes Checks for pattern match Pushes the input tokens onto output edges only if there is a pattern match Else no tokens are transmitted FG 1-4 ECALout 1-4 HCALout 1-4

18. 18/16 Why we need Unit Testing Throughput Time Throughput Time

19. 19/16 Test Folders (cont) Every test folder contains: »Input files that has the same name of the input port. »Output files that will be generated bye the test. »A runme.bat file that will run the simulator. As the test_creator is generic, the input files must contain the type of the data to be parsed correspondingly.