1. MicroTESK: An Extendable Framework for Test Program Generation Alexander Kamkin, Tatiana Sergeeva, Andrei Tatarnikov, Artemiy Utekhin {kamkin, leonsia, andrewt, utekhin}@ispras.ru Institute for System Programming of the Russian Academy of Sciences (ISPRAS) http://hardware.ispras.ru SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

2. CPU design Hardware Description Languages (HDL)  Verilog  VHDL SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

3. CPU verification Target CPU (HDL) lui s1, 0xdead ori s1, s1, 0x0 lui s3, 0xbeef ori s3, s3, 0xf add v0, a0, a2 sub t1, t3, t5 add t7, s1, s3 Test programs (assembler) Reference simulator (C/C++) Execution traces (formatted text) 0x2000: lui... 0x2004: ori... 0x2008: ori... 0x200c: lui... 0x2010: add... 0x2014: sub... 0x2018: add... Trace comparator (Perl, Python) 0x2000: lui... 0x2004: ori... 0x2008: ori... 0x200c: lui... 0x2010: add... 0x2014: sub... 0x2018: add... SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

4. Test program generation Random (RAVEN) Combinatorial (MicroTESK v1) Template-based (Genesys-Pro) Model-based (research projects) Requirements Design ? Test programs SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

5. Framework ? SYRCoSE 2013 – May 30, 2013 – Kazan, Russia Model-based Random Template-based Combinatorial

6. Modeling Library MicroTESK framework Modeling Framework Testing Framework Translator Coverage Extractor Design Library Test Template Processor Testing Library Coverage Library Constraint Solver Engine Specifications Test Templates Test Programs External Solvers Test Sequence Generators Test Data Generators Model Generator Coverage ModelDesign Model Model

7. MicroTESK plugin Design Aspect Instruction Set Memory Management Core Extensions Framework Plugin Translator Solvers Library Test Generators Modeling Testing SYRCoSE 2013 – May 30, 2013 – Kazan, Russia Pipelining User Defined

8. Specification example op ADD(rd:GPR, rs:GPR, rt:GPR) action = { if(NotWordValue(rs) || NotWordValue(rt)) then UNPREDICTABLE(); endif; tmp_word = rs ::rs + rt ::rt ; if(tmp_word != tmp_word ) then SignalException("IntegerOverflow"); else rd = sign_extend(tmp_word ); endif; } syntax = format("add %s, %s, %s", rd.syntax, rs.syntax, rt.syntax) op ALU = ADD | SUB | … Precondition Test Situations Equivalence Classes SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

9. Template example # Assembly -Style Code add r[1], r[2], r[3] sub r[1], r[1], r[4] # Ruby Control Statements (1..3).each do |i| add r[i], r[i+1], r[i+2] sub r[i], r[i], r[i+3] end # Test Sequence Block block (:engine => ”random”, :count => 2013) { add r[1], r[2], r[3] sub r[1], r[2], r[3] # Test Situation Reference do overflow end } SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

10. Test sequence generator # Test Sequence Block block (:combine => ”product”, :compose => ”random”) { # Nested Block A block (:engine => ”random”, :length => 3, :count => 2) { add r[a], r[b], r[c] sub r[d], r[e], r[f] mult r[g], r[h] div r[i], r[j] } # Nested Block B block (:engine => ”permutate” ) { ld r[k], r[l] st r[m], r[n] } } # Combination (1, 1) sub r[d], r[e], r[f] # Block A ld r[k], r[l] # Block B div r[i], r[j] # Block A st r[m], r[n] # Block B add r[a], r[b], r[c] # Block A # Combination (1, 2) st r[m], r[n] # Block B sub r[d], r[e], r[f] # Block A ld r[k], r[l] # Block B div r[i], r[j] # Block A add r[a], r[b], r[c] # Block A # Combination (2, 1) mult r[g], r[h] # Block A ld r[k], r[l] # Block B add r[a], r[b], r[c] # Block A st r[m], r[n] # Block B # Combination (2, 2) mult r[g], r[h] # Block A st r[m], r[n] # Block B mult r[g], r[h] # Block A ld r[k], r[l] # Block B add r[a], r[b], r[c] # Block A SYRCoSE 2013 – May 30, 2013 – Kazan, Russia Combinator Compositor

11. Constraint solver engine Collection of solvers:  Universal  Custom SYRCoSE 2013 – May 30, 2013 – Kazan, Russia MicroTESK framework uses Java Constraint Solver API http://forge.ispras.ru/projects/solver-api

12. Conclusion Framework architecture Support for common generation techniques Extendibility with other methods Flexible open-source environment SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

13. Contact information Institute for System Programming of RAS (ISPRAS) http://www.ispras.ru MicroTESK Test Program Generator http://forge.ispras.ru/projects/microtesk http://forge.ispras.ru/projects/microtesk Tatiana Sergeeva leonsia@ispras.ru SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

14. Thank you! Questions? SYRCoSE 2013 – May 30, 2013 – Kazan, Russia