1. Vendor Independent SEE Mitigation Solution For FPGAs Kamesh Ramani Pravin Bhandakkar Darren Zacher Melanie Berg (MEI – NASA Goddard)

2. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Agenda n Vendor Independent solution — Flow — Advantages — TMR Techniques — Handling of special cases — DRC and max fanout violation — Constraint handling — Formal Verification interface — User controls

3. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Vendor Independent Flow RTL Regular Synthesis Constraints Switches TMR TMR Options PNR TMR related synthesis controls EDIF SelectTechnology Regular options Switch off Netlist Optimizations Control inference of ROM/RAM, Shift Registers, DSPs Based on Scheme perform TMR on design and check for DRC rules

4. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Inference of Safe FSMs TMR Recognize TMR related attributes In the RTL Control not to infer Distributed RAM Constraints handling infrastructure Recognizing Combinational loops Block RAM inference and resource control DSP Inference, resource control No MAC inference No Counter inference Control on Flop absorption Formal Verification infrastructure SRL inference can be controlled SEU related controls during synthesis TMR TMRDRCProcessingTMR Max Fanout Processing Post TMR Processing TMRRelatedPNRoptions TMRPNRInteraction Synthesis From RTL to EDIF

5. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Vendor Independent Flow n Regular flow from RTL to synthesized Netlist and then to PNR n TMR can be applied on any chosen technology and device n Various optimizations for RadHard protection can be incorporated during synthesis itself n Output can be formally verified — against RTL — against non-TMR netlist n Special mitigation solution during synthesis for — DSPs, Black Boxes, RAMs, SRLs

6. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Vendor Independent Flow Advantages n Formally verifiable against RTL and non TMR netlist n Can be applied on any FPGA vendor chip n Can be applied on newer technology FPGAs seamlessly n Control throughout the synthesis flow to perform radiation related optimizations and choices n Controls at module level is available n Special handling for different technology cells n Ability to plug in dedicated RadHard modules for different inferred components n Can fix DRC and maxfanout violations effectively after TMR n Can seamlessly generate appropriate constraints, attributes, area and frequency reports for TMR netlist

7. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Voters n Voters — We define voters to be circuits that take in the output from triplicated flops and resolve it based on either majority or minority — We choose to use majority voters — Equation: A×B + B×C + C×A — Voters utilize combinatorial cells specific to target technology n LUTs in SRAM-based devices n MAJ3 in antifuse-based devices

8. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Various TMR Strategies - LTMR n Local TMR (LTMR) [1] — Triplicate sequential elements only, and majority vote the outputs n Flip-flops, shift registers, block RAMs, and sequential DSPs — The input data, control signals and clock will be shared by the triplicated flops — Reduces SEE occurrence to frequency dependent SET capture; clock trees, global routes and IOs are still susceptible [1] M. Berg, “Design for Radiation Effects, ”Invited Talk Presented 2008 at Military and Aerospace Programmable Logic Design, MAPLD, Annapolis, MD, September. 2008

9. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Various TMR Strategies - LTMR CombLogic Voter Voter Voter LTMR

10. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Various TMR Strategies - DTMR n Distributed TMR (DTMR) [1] — Apply TMR on sequential and combinational logic n Triplicate sequential and combinatorial logic; global routes and I/O are not triplicated n Vote out the triplicated logic just after the sequential elements n Triplicate the majority voting circuit as well to protect SET effects on the voting circuit — Reduces SEE occurrence; clock trees, global routes and IOs are still susceptible — Preferrable scheme for SEU and SET protection of technologies with hardened clock trees

11. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Various TMR Strategies - DTMR CombLogic DTMR Voter Voter Voter Voter Voter Voter Voter Voter Voter

12. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Various TMR Strategies - GTMR n Global TMR (GTMR) [1] — Apply TMR on the entire design including global buffers n Triplicate the sequential elements, combinational logic, voters and the global buffers n Voters converge the triplicated flop outputs at clock and control domain crossovers — This gives very high level of radiation protection — This scheme requires that the triplicated global lines have minimum skew between them — Preferred scheme for commercial SRAM based FPGAs

13. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Various TMR Strategies - GTMR CombLogic GTMR Voter Voter Voter Voter Voter Voter Voter Voter Voter

14. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Special Case Handling Overview n In vendor independent flow we can handle embedded resources effectively — Tech cells such as RAMs, DSPs, Shift registers etc — Need to triplicate and vote datapath can limit usage — Ability to control automated embedded resource inference n To Selectively infer n To infer with restricted features — Synthesis tool decides embedded resource allocation earlier in the implementation flow n Enables more effective TMR application. — Gives better control over synthesis for radiation hardening

15. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Embedded Resource Handling n Embedded RAM — Triplicated and voters are inserted at each output — Treated as sequential elements and will be TMRed in all the schemes — The user can instantiate an error correcting (EDAC) RAM n Supply black box or netlist IP, or set an attribute on the instance n Instance will be treated like a black box or IP and not triplicated

16. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Embedded Resource Handling n Embedded Shift Registers (e.g. SRL) — By default, embedded shift registers will not be inferred during synthesis n User has option to enable shift register inference if desired — If present in the design, embedded shift registers are triplicated and a voter is inserted at the output

17. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Embedded Resource Handling n DSP — DSPs can be classified for TMR broadly into n Sequential DSPs n Combinational DSPs — Sequential DSPs n Contain Flops in them — Flops can be at the input, output or as pipeline registers n Infer DSPs with only flops at the output by default n These DSPs will be triplicated and voters inserted at every output n Scan chain facilities in these DSPs will not be used as we cannot insert voters at their outputs

18. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Embedded Resource Handling n DSPs — Combinational DSPs n These contain only combinational logic n Will be triplicated and voted out in DTMR and GTMR schemes only — Multiply-Accumulate (MAC) n MACs will not be inferred — They contain loops and can potentially get stuck after a fault — Hence the loop will be outside the DSP so that voters can be inserted in the loop to correct SEUs

19. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Special Case Handling n I/O Boundary Flops — Flops at input and output may need special handling based on the TMR scheme — LTMR n The inputs to the TMRed design will fanout to triplicated flop instances n The output flops will also have a voter at its output similar to other flops in the design. n If user wishes, by using regular synthesis attributes can absorb the flops into the pads — Flop not triplicated when absorbed into the pad

20. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Special Case Handling I/O Boundary Flops n DTMR and GTMR — The inputs to the TMRed design will fanout to triplicated design — The outputs from the TMRed design will converge at output flops — Voters need to be applied at the output flops to converge the output — If user wishes, by using regular synthesis attributes can absorb the flops into the pads n Flop not triplicated when absorbed into the pad — Can choose to triplicate pins on top level n Each of the triplicated outputs can be voted n Or user can choose to absorb flops, without voting, into pads using normal synthesis attributes

21. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Special Case Handling n Boxes — By boxes we mean black, white, grey, clear etc — Need to be mitigated by the user, as there is either no or limited visibility inside them — All inputs to the box will converge at the box input boundary n A voter will be inserted before each box input — All outputs from box will fanout to triplicated instances — Tool will fix and report any max fanout violation at box outputs

22. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Special Case Handling n Latches — Latches are treated similar to sequential elements — If a latch is present we will always triplicate it and insert voters at the output n This is because latches contain loops which have the ability to retain faults

23. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Special Case Handling n Combinational Loops — Combinational loops should be avoided in a design targeted for mitigation — However if combinational loop is present, we will insert a voter at the point of feed back — We will also warn the user about the combinational loop as well

24. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Special Case Handling n Clock generation circuits — Clock generation circuits such as PLLs, DCM etc are susceptible to SEEs and should be avoided — We warn the user about these circuits — We do not triplicate these, nor vote them out

25. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Design Rule Check (DRC) violations n TMR on some cells can cause DRC violations — Inserting voters between cascade pins of embedded resources is generally not allowed n Block RAM cascading n DSP cascading n Scan chain feature of DSPs — The violations are handled by n not cascading the embedded resources or n using the appropriate non-cascade input and output pins of those resources — Global buffers such as those with pads cannot be triplicated n To triplicate them we split them into pad cell and buffer cell n Buffer cells are then triplicated — Flow advantage: preventive steps can be taken during synthesis

26. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Max fanout violations n Potential TMR violations occur at: — Input nets that feed triplicated logic — Black box outputs — Flop inputs in LTMR n Flow advantage: Since TMR is part of the synthesis flow such violations are detected and fixed

27. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Constraints Handling n Constraints on original instance copied to triplicated instances n Constraints on flop output is transferred to voter output(s) n Appropriate constraints are written out at the end of synthesis for the TMRed design as a whole n Flow advantages: — Constraints are applied seamlessly throughout the TMRed design — No need of post processing constraints after synthesis

28. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Formal Verification RTL vs. TMR Netlist n Novel approach regarding TMR insertion — “Formal Verification of Advanced Synthesis Optimizations”, Anant Kumar Jain et al, MAPLD-09 n Formal Verification Interface (FVI) constraints for triplicated instances generated — This assists FormalPro in verifying the generated netlist against RTL n Flow advantages: — FVI constraints are automatically generated during synthesis — Post-TMR netlist can be easily verified

29. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Formal Verification TMR Netlist vs. Non TMR Netlist n FVI matching rules are generated during synthesis — Matching rules helps to identify instance in normal netlist with its triplicated counterparts in the TMRed netlist n Flow advantages: — FVI matching rules are automatically generated during synthesis — Post-TMR netlist can be easily verified

30. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Module Level User Controls n Ability to apply different TMR techniques on different modules n Allows TMR application on an as-needed basis for a given module — Voters converge the triplicated flop outputs on the module boundaries with lesser TMR protection n Using attributes can specify TMR scheme on a — given instance in RTL — On all instances of a given module in RTL n TMR scheme is inherited through the hierarchy n Flow advantages: — Control at the RTL level — Beneficial for design review — Greater flexibility to the user

31. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Additional User Controls n Voter insertion — Force Voter Insertion on net or net’s driver’s output if specified — Can be specified using an attribute n Triplication control — Specify a given instance not be triplicated — Can be specified using an attribute

32. MAPLD 2009 - Vendor Independent SEE mitigation solution for FPGAs Summary n Flow implemented as part of Precision synthesis tool n TMR output netlist formally verifiable against RTL and non TMR netlist n TMR can be applied on any FPGA vendor chip n TMR can be applied on newer technology FPGAs seamlessly n Controls available throughout the synthesis flow to perform mitigation related optimizations and choices n TMR netlist is free of any DRC and maxfanout violations n Constraints, attributes, area and frequency reports for TMR netlist are automatically generated.