1. Detecting Hardware Trojans in Unspecified Functionality Using Mutation Testing Nicole Fern K.-T. Tim Cheng UC Santa Barbara 1

2. Main Contributions Information leakage Trojan only modifying unspecified functionality Mutation testing based detection method Discovered vulnerabilities and verification holes in UART design with sophisticated testbench 2

3. Hardware Trojans Malicious circuitry inserted in the hardware design Can be inserted by any party with access to the design! Goals: leak information, induce faults, chip failure, gain root privileges, etc. 3

4. Trojan Classes 1.The logic functions of some design signals are altered, system specifications are violated 2.The Trojan leaks information through side- channels 3.The logic functions of only those design signals which have unspecified behavior are altered to add malicious functionality without violating system specifications 4

5. FIFO Example What is the value of read_data when read_en is 0? 5

6. Threat Model Trojans can be inserted in the RTL and all subsequent design stages Our method analyzes RTL code, identifies Trojans which leak information from the circuit by hiding in unspecified functionality Behavior of circuit under Trojan activation condition is unspecified and unverified – Trojan can be active frequently, yet go undetected! 6

7. Detection Methodology Overview Goal: design independent method to identify dangerous unspecified functionality Use mutation testing to uniformly sample possible design modifications (can think of as very simple Trojan modifications) Use additional information to determine if modification is “dangerous” 7

8. Mutation Testing Basic Idea: If the testbench cannot detect an artificial error, testbench likely incapable of detecting a real error 8 DUT Tests Checker Detected Undetected Add more tests Fix

9. Mutation Testing Used in software domain since the 1970’s to test program correctness – Can also identify security weaknesses 1,2 Used in hardware domain for testbench qualification 3,4 Long simulation runtime and manual effort required for mutant analysis are drawbacks 1.Jia and Harman. An analysis and survey of the development of mutation testing. TSE, 2011. 2.Breech et al. An attack simulator for systematically testing program-based security mechanisms. ISSRE, 2006. 3.Bombieri et al. Functional qualification of TLM verification. DATE, 2009. 4.Lisherness et al. Mutation Analysis with Coverage Discounting. DATE, 2013. 9

10. Interpreting Undetected Faults 2 Classifications: 1.Affect Poorly Tested Functionality Ex. Interrupt line set to static 0 2.Redundant Fault: does not affect design functionality Ex. for (int i = 0; i < 10; i++) Ex. Adder output toggled only during intermediate cycles 10 Coverage Discounting: automated analysis to identify Class 1 faults 1.Lisherness et al. Mutation Analysis with Coverage Discounting. DATE, 2013. !=

11. Identifying Dangerous Faults Attacker-observable signals: primary outputs, software-visible registers, network interface, bus interface,… Information leakage possible if undetected fault causes change in attacker-observable signals 11 if (key) { code w/ fault; } else { original code; } key 0 1 0/1 1/0 Attacker- observable signal

12. Dangerous Unspecified Functionality Undetected Fault Classes: 1.Affect Poorly Tested Functionality Ex. Interrupt line set to static 0 2.Redundant Fault: does not affect design functionality Ex. for (int i = 0; i < 10; i++) Ex. Adder output toggled only during intermediate cycles 12 != Undetected Fault Classes: 1.Affect Poorly Tested Functionality Ex. Interrupt line set to static 0 2.Affect attacker-observable signals, but not design functionality 3.Causes no change in any signal values (truly redundant) Automated method to identify only dangerous Class 2 faults

13. Trojan Detection Methodology 13 DUT Tests Checker Detected Undetected Add more tests Fix Before Fault Injection Record functional coverage Record attacker-observable signals Functional coverage differs? Fault affects poorly tested specified functionality Attacker- observable signals differ? Fault affects dangerous unspecified functionality Refine Specification

14. Poorly Tested Specified v. Dangerous Unspecified Functionality Redundant 14

15. Methodology Applied to FIFO Example What is the value of read_data when read_en is 0? 1)Fault is undetected 2)Causes changes in attacker-observable signal read_data! 15 ||

16. Fault Ranking Mechanism Might be too expensive to analyze all faults classified as dangerous Ideal to identify and fix functionality related to the “most dangerous” faults first Quantities easy to measure for each fault: 1.Number of attacker-observable bits differing 2.Total time attacker-observable signals differ 3.Number of distinct tests producing differences in attacker-observable signals 16

17. UART Controller Case Study OpenCores IP, OVM testbench from EDA vendor Verification Infrastructure: 80 tests, 846 coverpoints Mutation Testing: 1183 total faults injected – 110 faults not detected 32 caused differences in attacker-observable signals – 4 discounted coverpoints UART wb_dat_o int_o wb_ack_o baud_o stx_pad_o rts_pad_o dtr_pad_o Output signals going to main processor Output signals for serial data transmission 17 38 attacker-observable bits

18. Wishbone Bus Trojan Analyzed 3 most dangerous faults – All affect bus between UART and main processor – All affect output enable signal for data bus Information can be leaked on data bus if a valid read transaction is NOT occurring! UART wb_dat_o int_o wb_ack_o baud_o stx_pad_o 18 rts_pad_o dtr_pad_o

19. Wishbone Bus Trojan The 3 Undetected Faults: Write enable is de- asserted (read transaction) Slave is selectedValid bus cycle in progress | | | | | | Undetected Faults changing & to | 19

20. Wishbone Bus Trojan Data can be leaked on wb_dat_o whenever all 4 conditions for a valid read transaction are not simultaneously met! | | We will leak data when wbstate==01 but not during a valid read transaction 20

21. Improving Bus Checker Data during invalid bus cycle, or output data during write cycles was not checked New assertion added: value of wb_dat_o cannot change unless design has been reset or read request is being acknowledged New check detects 3 faults and bus Trojan 21 Method able to highlight unspecified functionality in on-chip bus protocols allowing attacker to leak information using the system bus

22. Interrupt Functionality 5 possible interrupt sources, 1 interrupt bit Next most dangerous fault causes int_o to become X for many cycles during 49 of 80 tests! Attacker can spuriously change int_o to encode information UART wb_dat_o int_o wb_ack_o baud_o stx_pad_o 22 rts_pad_o dtr_pad_o

23. Why was this fault undetected? Interrupt Identification Register (IIR) reveals source of interrupt For all 5 events which cause interrupts, testbench checks that IIR is properly set, and that int_o is set within 10 cycles Due to a bug in the testbench if int_o is X this check is skipped! 23

24. Interrupt Functionality Functional coverage did not change, suggesting that the coverage model is insufficient! 24 Method able to highlight verification hole relating to specified interrupt functionality

25. Conclusions Illustrated how unspecified functionality can be altered for malicious purposes Automated analysis methodology: 1.Uncovered Trojan leaking information through unspecified functionality in a standard bus protocol 2.Identified verification hole for interrupt functionality 25

26. Questions? 26

27. Payload violates Design Specification Relies on rare triggering conditions Detection likely if triggered during verification/testing Detection methods identify “almost unused logic” Ex. When specific 128-bit plaintext value appears, dump key bits to output Payload operates completely within unspecified functionality Design behavior unspecified for activation conditions Likely not detected if triggered during testing Functionality Trojan modifies is unspecified and unverified Activation condition can occur frequently during testing (ex. read_en == 0) 27 Comparison of Trojans in Logic Domain

28. Coverage Discounting DUT Tests Checker Detected Undetected Add more tests No Yes Fix Coverage Changed? 28