Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Institute for System Programming of the Russian Academy of Sciences (ISPRAS) Summer School on Software Engineering and Verification (SSSEV) July 17-27, Moscow, Russia

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 2 of 50 Agenda Introduction  Digital hardware design  Simulation-based verification  Time abstraction in hardware modeling Main part  Time abstraction levels  Model-based reaction checking  Error diagnostics Conclusion  C++TESK testing toolkit  Future work  Questions & answers

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 3 of 50 Time Time is a part of measuring system used to sequence events, to compare the durations of events and the intervals between them… Wikipedia The only reason for time is so that everything doesn’t happen at once Albert Einstein

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 4 of 50 Abstraction Abstraction is the act of considering something as a general quality or characteristic, apart from concrete realities, specific objects, or actual instances Webster’s Dictionary Abstraction captures only those details about an object that are relevant to the current perspective Wikipedia Time abstraction is (1) generalization of events ordering relationship and (2) factorization of time intervals between them This Presentation

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 5 of 50 Hardware design in a nutshell Hardware is designed using hardware description languages (HDL), like Verilog and VHDL The result is a software model that can be executed in an HDL simulator The main approach to verify a design is to test the HDL model (simulation-based verification) To automate simulation-based verification, reference models are used (C/C++)

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 6 of 50 Inputs, outputs, and system clock Inputs Outputs Clock

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 7 of 50 Hardware description language (HDL) input S; output R1, R2; void design() { while(true) { wait(S); delay(6); R1 = 1; delay(1); R1 = 0; delay(1); R2 = 1; delay(1); R2 = 0; V1 = 1; }} CLK S R1 6 cycles R2 Concurrent assignments

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 8 of 50 Simulation-based verification S2 R1 R2 Stimuli Reactions S3S1 R3 Generation Checking

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 9 of 50 Stimuli generation Reaction checking Coverage tracking Simulation-based verification tasks Coverage Tracker Reaction Checker Stimulus Generator Stimulus Generator Target Design Reaction Checker Test Coverage

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 10 of 50 Reaction checking Number of reactions is correct Each reaction is correct Order of reactions is correct Time intervals between reactions are correct Timing Functionality

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 11 of 50 Design modifications Timing Interface Function Requirements are not time-accurate; design’s timing constantly changes 

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 12 of 50 Model abstraction Abstract models are easier to develop and to maintain Abstract models are more stable (reusable) Abstract models are less error-prone Abstract models provide lower verification quality Abstract models are less deterministic and predictable ++++++

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 13 of 50 Time abstraction S R1 R2 Stimulus Reactions Events Concrete event sequence S #6 R1 #2 R2 Abstract specification S #+ R1 #* R2 More abstract specifications S ((#+ R1 #* R2) | (#+ R2 #* R1)) S ((#+ R1) || (#+ R2)) S R1 R2 # S R1 R2 # S R1 R2 # S R1 R2 # S R1 R2 # S # R1 R2 # S R1 R2 # #

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 14 of 50 Time abstraction in practice Hardware design modeling  Development of reference models at different abstraction levels (specification of time properties)  Change of abstraction level (refinement of time properties) Reference model adaptation  Adaptation of abstract (untimed) reference models for co-simulation in a time environment  Tuning time properties being checked without changing a reference model (reaction arbitration, etc.)

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 15 of 50 To be continued… Questions?

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 16 of 50 Agenda Introduction  Digital hardware design  Simulation-based verification  Time abstraction in hardware modeling Main part  Time abstraction levels  Model-based reaction checking  Error diagnostics Conclusion  C++TESK testing toolkit  Future work  Questions & answers

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 17 of 50 Time abstraction levels Time-accurate (cycle-accurate) models … Time-inaccurate (time-approximate) models … Untimed (functional) models

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 18 of 50 input bool val_in_data; input uint8_t in_data; output bool val_out_data; output uint8_t out_data; void store_word() { uint32_t data = 0; uint32_t temp = 0; while(true) { wait(val_in_data); for(int i = 0; i < 4; i++) { data |= in_data << (i << 3); delay(1); } temp = memory; delay(1); memory = data; delay(1); val_out_data = 1; for(int i = 0; i < 4; i++) { out_data = (temp >> (i << 3)) & 0xff; delay(1); } val_out_data = 0; }} input bool val_in_data; input uint8_t in_data; output bool val_out_data; output uint8_t out_data; void store_word() { uint32_t data = 0; uint32_t temp = 0; for(int i = 0; i < 4; i++) { data |= in_data << (i << 3); delay(1); } temp = memory; delay(1); memory = data; delay(1); val_out_data = 1; for(int i = 0; i < 4; i++) { out_data = (temp >> (i << 3)) & 0xff; delay(1); } val_out_data = 0; } Cycle-accurate models

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 19 of 50 Modeling concurrency Time delay(1) Operation delay(1) return Operation delay(1) return delay(1) Operation delay(1) return delay(1) return delay(1) Operation

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 20 of 50 uint32_t store_word(uint32_t data) { uint32_t temp = memory; memory = data; return temp } Functional (untimed) models: time interval abstraction input in_iface ; output out_iface ; void store_word() { uint32_t temp = memory; memory = recv(in_iface); // delay([0,  )) = #* send(out_iface, temp); }

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 21 of 50 input bool val_in_data[ ]; input uint8_t in_data[4]; output bool val_out_data[ ]; output uint8_t out_data[4]; void store_word() { uint32_t data = 0; uint32_t temp = 0; for(int i = 0; i < 4; i++) { data |= in_data[t 1 ] << (i << 3); delay(1); } temp = memory; delay(1); memory = data; delay(1); val_out_data = 1; for(int i = 0; i < 4; i++) { out_data[t 2 ] = (temp >> (i << 3)) & 0xff; delay(1); } val_out_data = 0; } Functional (untimed) models (cont.) t 1 ++ t 2 ++ input in_iface ; output out_iface ; void store_word() { uint32_t data = 0; uint32_t temp = 0; data = recv(in_iface); temp = memory; memory = data; send(out_iface, temp); }

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 22 of 50 Functional (untimed) models: events ordering abstraction input in_iface ; output out_iface1 ; output out_iface2 ; void store_word() { uint32_t temp = memory; memory = recv(in_iface); // Order of events is undefined send(out_iface1, temp); send(out_iface2, memory); }

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 23 of 50 Time-approximate models input in_iface ; output out_iface ; // Reactions ordering void store_word() { uint32_t data = 0; uint32_t temp = 0; data = recv(in_iface, in_data); temp = memory; delay([0, 3]); // Delays are approximate memory = data; delay([1, 4]); // Delay=(0+1)=1, Timeout=(3-0)+(4-1)=6 send(out_iface, temp); }

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 24 of 50 Transaction-level modeling (TLM) TLM is a hardware modeling approach that separates communication among design units from the functional description of those units Discrete signals distributed in time Data Wires/pins Package Data Channels/interfaces Untimed data package (message) TLM is data transmission encapsulation

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 25 of Model interface adapters Target Design (HDL Model) Input interface #1 Input interface #N Data Output interface #1 Output interface #M Reaction Checker Input Interface Adapters (Serializers) Output Interface Adapters (Deserializers) Reference Model (TLM) input in_iface ; output out_iface ; void store_word() { uint32_t temp = memory; memory = recv(in_iface);... send(out_iface, temp); }

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 26 of 50 Problems caused by time abstraction Design state uncontrollability  Design| Model is not deterministic  Problems in stimulus generation & coverage tracking Reaction order ambiguity  Order of reactions is unpredictable  Problems in reaction checking

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 27 of 50 Design state uncontrollability S R1R2 Design’s Inputs/Outputs Model’s State S’ Nondeterministic behavior Design’s State Uncontrollable actions Pre Impl (S’)=false Pre Impl (S’)=true

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 28 of 50 Reaction order ambiguity S R2R1 Design’s Inputs/Outputs recv(in_iface, S); Model Execution Trace send(out_iface, R1); send(out_iface, R2);... Failed: R2  R1 Different order Output Interface’s Queue R1R2 Passed: R2  Queue

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 29 of 50 Agenda Introduction  Digital hardware design  Simulation-based verification  Time abstraction in hardware modeling Main part  Time abstraction levels  Model-based reaction checking  Error diagnostics Conclusion  C++TESK testing toolkit  Future work  Questions & answers

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 30 of 50 To be continued… Questions?

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 31 of 50 Reaction arbitration Reaction arbiter finds a model reaction corresponding to a reaction received from the target design Reaction checking accuracy depends not only on the model abstractness, but on reaction arbitration as well Each output interface has its own reaction arbiter Reaction arbiters encapsulates all reaction ordering aspects of the reaction checker

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 32 of 50 Model-based reaction checker Target Design Reaction Checker Reaction Comparators Reference Model Reaction Arbiters Input Interface Adapters Output Interface Adapters Stimuli Design’s Reactions Model’s Reactions Verdict

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 33 of 50 Reaction arbiter types Deterministic model-based arbiters arbiter: 2 Reaction  Reaction  {fail} Adaptive arbiters arbiter: 2 Reaction  Feedback  Reaction  {fail} Two-level arbiters arbiter(reactions)  arbiter 2 (arbiter 1 (reactions), feedback)  Nondeterministic model-based arbiter arbiter 1 : 2 Reaction  2 Reaction : arbiter 1 (reactions)  reactions  Adaptive arbiter arbiter 2 : 2 Reaction  Feedback  Reaction  {fail}

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 34 of 50 Deterministic model-based arbiter R1 Design’s Reactions Model’s Reactions send(R1); send(R2);... R1R2 Reaction Arbiter R1 R2 FIFO  ✕ Comparison SR Order is defined

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 35 of 50 Adaptive arbiter R1 Design’s Reactions Model’s Reactions send(R1); send(R2);... R1 R2 Reaction Arbiter R1 R2  ✕ Get(R1) Comparison SR Order is undefined Feedback

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 36 of 50 Two-level arbiter R1 Design’s Reactions Model’s Reactions send(R1); send(R2);... R1 R2 Arbiter #1 R1 R2  ✕ Get(R1) Comparison SR Order is partially defined Arbiter #2 Feedback Candidates

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 37 of 50 Reaction checking algorithm On model reaction R on interface out: Reactions out := Reactions out  {R} wind(Timer R ) On model reaction’s time-out: return “Missing reaction” On design’s reaction R’ on interface out: Candidate out := Reactions out if(|Candidate out |  2) { Candidate out := Arbiter 1 out (Reactions out ) if(|Cadidates out |  2) Candidates out := Arbiter 2 out (Reactions out, R’); } assert(|Cadidates out | < 2) if(Cadidates out =  ) return “Unexpected reaction” if(R’  get(Candidates out ))) return “Incorrect reaction”

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 38 of 50 Simple error classification “Missing reaction” The reference model generates a reaction, but the design’s reaction is not appeared in time “Unexpected reaction” The target design produces a reaction, but it is not expected by the reference model “Incorrect reaction” Both the reference model and the target design generate reactions, but those reactions are different

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 39 of 50 Classification of abstraction levels  Cycle-accurate models G (  out  |Reactions out | < 2)  Cycle-accurate models (Time(R) = 0)  Quasi cycle-accurate models (otherwise)  Order-accurate models G (  out  |Reactions out | < 2  |Arbiter 1 out (Reactions out )| < 2)  Order-accurate models (Arbiter 1 out = FIFO)  Quasi order-accurate models (otherwise) Order-inaccurate models otherwise

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 40 of 50 Error diagnosis problem 0x19c3827ab2920e58  0xf953e8d83a9b9209 0xf953e8d83a9b9209  0x19c3827ab2920e58

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 41 of 50 Error diagnosis approach  ●,○   ○,   ○,○   ○,●   ○,   ●,● ,□   ●,○  ,○   □,○   ○,○   ●,●   ●,■   ○,■   ■,■   □,□   ■,■   ○,●  ●,○ 

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 42 of 50 Control errors model ,  =   ○,○   ,   ○,●  +  ●,○    ○,○  +  ●,●   ○,■  +  ●,○    ○,○  +  ●,■   ○,  + ,○    ○,○  + ,   ○,●  +  ●,    ○,  +  ●,●   ○,●  + ,○   ,●  +  ○,○ 

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 43 of 50 Functional errors model  ○,□    ○,○   ○,■  +  ●,□    ○,□  +  ●,■   ○,  + ,□    ○,□  + ,   ○,■  +  ●,    ○,  +  ●,■   ○,●  + ,□   ,●  +  ○,□ 

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 44 of 50 Agenda Introduction  Digital hardware design  Simulation-based verification  Time abstraction in hardware modeling Main part  Time abstraction levels  Model-based reaction checking  Error diagnostics Conclusion  C++TESK testing toolkit  Future work  Questions & answers

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 45 of 50 C++TESK testing toolkit Development of hardware models at different abstraction levels and model adapters Description of test coverage and test scenarios Report generation (coverage and errors) Automated test sequence generation based on state graph exploration Test execution parallelization based on distributed state graph exploration

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 46 of 50 C++TESK testing toolkit (cont.) Web:

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 47 of 50 Conclusion Time abstraction hides control logic (timing) of a design (pipelining, arbitration, queuing, etc.) Time-abstract models are easier to develop and sufficiently easier to maintain (timing is changeable) Time-abstract models reduce verification efforts and allow creating reusable tests (quality is reduced also) Verification quality can be increased by refining time properties of a model (events ordering, durations, etc.)

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 48 of 50 Conclusion (cont.) Interface transformation  serialization(S): S  inputs  deserialization(R’): outputs  R’ Reaction queuing  send(R) is asynchronous: enqueue(R) Reaction arbitration  arbiter 1 (queue)  candidates  arbiter 2 (candidates, R’)  R   R, R’  Reaction comparison  compare(R, R’)  status Error diagnosis  diagnose({  R i, R i ’  } i=1,n )  diagnosis

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 49 of 50 Contacts Institute for System Programming of RAS (ISPRAS) Hardware Verification ISPRAS Alexander Kamkin

Time Abstraction in Simulation-Based Hardware Verification Alexander Kamkin Summer School on Software Engineering and Verification (SSSEV) - July 17-27, Moscow, Russia 50 of 50 The End Thank you! Questions?