MicroTESK: An Extendable Framework for Test Program Generation Alexander Kamkin, Tatiana Sergeeva, Andrei Tatarnikov, Artemiy Utekhin {kamkin, leonsia, andrewt, Institute for System Programming of the Russian Academy of Sciences (ISPRAS) SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

CPU design Hardware Description Languages (HDL)  Verilog  VHDL SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

CPU verification Target CPU (HDL) lui s1, 0xdead ori s1, s1, 0x0 lui s3, 0xbeef ori s3, s3, 0xf add v0, a0, a2 sub t1, t3, t5 add t7, s1, s3 Test programs (assembler) Reference simulator (C/C++) Execution traces (formatted text) 0x2000: lui... 0x2004: ori... 0x2008: ori... 0x200c: lui... 0x2010: add... 0x2014: sub... 0x2018: add... Trace comparator (Perl, Python) 0x2000: lui... 0x2004: ori... 0x2008: ori... 0x200c: lui... 0x2010: add... 0x2014: sub... 0x2018: add... SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Test program generation Random (RAVEN) Combinatorial (MicroTESK v1) Template-based (Genesys-Pro) Model-based (research projects) Requirements Design ? Test programs SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Framework ? SYRCoSE 2013 – May 30, 2013 – Kazan, Russia Model-based Random Template-based Combinatorial

Modeling Library MicroTESK framework Modeling Framework Testing Framework Translator Coverage Extractor Design Library Test Template Processor Testing Library Coverage Library Constraint Solver Engine Specifications Test Templates Test Programs External Solvers Test Sequence Generators Test Data Generators Model Generator Coverage ModelDesign Model Model

MicroTESK plugin Design Aspect Instruction Set Memory Management Core Extensions Framework Plugin Translator Solvers Library Test Generators Modeling Testing SYRCoSE 2013 – May 30, 2013 – Kazan, Russia Pipelining User Defined

Specification example op ADD(rd:GPR, rs:GPR, rt:GPR) action = { if(NotWordValue(rs) || NotWordValue(rt)) then UNPREDICTABLE(); endif; tmp_word = rs ::rs + rt ::rt ; if(tmp_word != tmp_word ) then SignalException("IntegerOverflow"); else rd = sign_extend(tmp_word ); endif; } syntax = format("add %s, %s, %s", rd.syntax, rs.syntax, rt.syntax) op ALU = ADD | SUB | … Precondition Test Situations Equivalence Classes SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Template example # Assembly -Style Code add r[1], r[2], r[3] sub r[1], r[1], r[4] # Ruby Control Statements (1..3).each do |i| add r[i], r[i+1], r[i+2] sub r[i], r[i], r[i+3] end # Test Sequence Block block (:engine => ”random”, :count => 2013) { add r[1], r[2], r[3] sub r[1], r[2], r[3] # Test Situation Reference do overflow end } SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Test sequence generator # Test Sequence Block block (:combine => ”product”, :compose => ”random”) { # Nested Block A block (:engine => ”random”, :length => 3, :count => 2) { add r[a], r[b], r[c] sub r[d], r[e], r[f] mult r[g], r[h] div r[i], r[j] } # Nested Block B block (:engine => ”permutate” ) { ld r[k], r[l] st r[m], r[n] } } # Combination (1, 1) sub r[d], r[e], r[f] # Block A ld r[k], r[l] # Block B div r[i], r[j] # Block A st r[m], r[n] # Block B add r[a], r[b], r[c] # Block A # Combination (1, 2) st r[m], r[n] # Block B sub r[d], r[e], r[f] # Block A ld r[k], r[l] # Block B div r[i], r[j] # Block A add r[a], r[b], r[c] # Block A # Combination (2, 1) mult r[g], r[h] # Block A ld r[k], r[l] # Block B add r[a], r[b], r[c] # Block A st r[m], r[n] # Block B # Combination (2, 2) mult r[g], r[h] # Block A st r[m], r[n] # Block B mult r[g], r[h] # Block A ld r[k], r[l] # Block B add r[a], r[b], r[c] # Block A SYRCoSE 2013 – May 30, 2013 – Kazan, Russia Combinator Compositor

Constraint solver engine Collection of solvers:  Universal  Custom SYRCoSE 2013 – May 30, 2013 – Kazan, Russia MicroTESK framework uses Java Constraint Solver API

Conclusion Framework architecture Support for common generation techniques Extendibility with other methods Flexible open-source environment SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Contact information Institute for System Programming of RAS (ISPRAS) MicroTESK Test Program Generator Tatiana Sergeeva SYRCoSE 2013 – May 30, 2013 – Kazan, Russia

Thank you! Questions? SYRCoSE 2013 – May 30, 2013 – Kazan, Russia