Formal Performance Evaluation of AMBA-based System-on-Chip Designs Gabor Madl1, Sudeep Pasricha1, Qiang Zhu2, Luis Angel D. Bathen1, Nikil Dutt1* 1 Center for Embedded Computer Systems, UC Irvine 2 Fujitsu Laboratories Limited 1{gabe, sudeep, lbathen, dutt}@ics.uci.edu, 2shu.kyou@jp.fujitsu.com *This research was partially supported by a CPCC Fellowship Emsoft 2006 November 12, 2018

Outline Motivation Model-based evaluation of SoCs Outline Motivation Formal modeling Functionality Case study Performance Conclusion Outline Motivation Model-based evaluation of SoCs Formal modeling of SoCs using the AMBA AHB bus Functional verification of AMBA-based SoC designs Case study: Digital camera SoC Performance evaluation of AMBA-based SoC designs Simulation-based performance evaluation Model checking-based performance evaluation Conclusion November 12, 2018

Challenges in Bus Protocol Specs Outline Motivation Formal modeling Functionality Case study Performance Conclusion Challenges in Bus Protocol Specs Bus protocols have to implement complex methods for component interactions Increasing number of (often contradictory) constraints There is a trend to propagate the management of bus constraints to the SoC designer Bus protocols have a critical role in providing a reliable SoC platform Specifications are written as a combination of natural languages and timing diagrams Effective way to explain the use of the protocol to SoC designers Cannot cover every possible use case Introduces ambiguity in the specification November 12, 2018

The Need for Functional Verification Outline Motivation Formal modeling Functionality Case study Performance Conclusion The Need for Functional Verification SoC designers have to use the specification as a guide to satisfy constraints in their design Which constraints are missing or implied from the specification? Different vendors may implement ambiguous specifications differently The interoperability of such IP blocks may be at risk Simulations and test vectors are widely used to evaluate SoC designs It can only show the presence of errors, not their absence Time consuming, limiting the analysis to a few test cases There is a need to verify the functionality of SoC designs even when well-known protocols have been used November 12, 2018

Formal Methods for SoC Evaluation Outline Motivation Formal modeling Functionality Case study Performance Conclusion Formal Methods for SoC Evaluation Simulations can accurately model some execution traces of a design Did I consider all the interesting execution traces? Formal methods can capture all execution traces of a design with some accuracy More abstract representation than for simulations How accurate are my results? How do I obtain parameters for the design? Combine simulations & formal methods to achieve best coverage and performance Provides for the best coverage possible with the highest accuracy for a given time frame November 12, 2018

Model-based Verification Outline Motivation Formal modeling Functionality Case study Performance Conclusion Model-based Verification Property Verification Design feedback Simulation Model Functional Verification Perf. Evaluation Input Formal Model Semantic mapping Domain Specific Model AnalysisDomain Simulations Parameters November 12, 2018

Formal Modeling of AMBA AHB Outline Motivation Formal modeling Functionality Case study Performance Conclusion Formal Modeling of AMBA AHB Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations Finite state machines – NuSMV tool Cycle-accurate bus model Arbitration delays Pipelining Busy slaves SPLIT and RETRY transfers – for functional verification only 2-cycle response times for RETRY and SPLIT responses We model AMBA masters using 6 states (idle, busreq, haddr, read, write, error) We model slaves using 4 states (idle, write, read, error) Round-robin arbiter (specific to AMBA) November 12, 2018

Functional Verification Domain-specific Model Outline Motivation Formal modeling Functionality Case study Performance Conclusion Model of an AMBA Slave Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations MODULE slave (HADDR, HTRANS, HWDATA, HRDATA, HREADY, HRESP, HMASTER, HSPLIT, MASK_MASTER1, MASK_MASTER2, MASK_MASTER3, SLAVE_STATE) VAR state : {idle, write, read, error}; prev state : {idle, write, read, error}; extended : boolean; ASSIGN init (state) := idle; init (prev_state) := state; init (extended) := 0; next (prev_state) := state; next (state) := case SLAVE_STATE != x : SLAVE_STATE; HRESP = SPLIT : idle; !HREADY : state; HTRANS = BUSY : state; HRESP = RETRY : prev state; state = idle & HTRANS = NONSEQ & HADDR : write; state = idle : state; state = write & HTRANS = NONSEQ : read; state = read & HTRANS = NONSEQ & HWDATA : idle; 1 : error; esac; ... November 12, 2018

Functional Verification Domain-specific Model Outline Motivation Formal modeling Functionality Case study Performance Conclusion Model of an AMBA Slave Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations MODULE slave (HADDR, HTRANS, HWDATA, HRDATA, HREADY, HRESP, HMASTER, HSPLIT, MASK_MASTER1, MASK_MASTER2, MASK_MASTER3, SLAVE_STATE) VAR state : {idle, write, read, error}; prev state : {idle, write, read, error}; extended : boolean; ASSIGN init (state) := idle; init (prev_state) := state; init (extended) := 0; next (prev_state) := state; next (state) := case SLAVE_STATE != x : SLAVE_STATE; HRESP = SPLIT : idle; !HREADY : state; HTRANS = BUSY : state; HRESP = RETRY : prev state; state = idle & HTRANS = NONSEQ & HADDR : write; state = idle : state; state = write & HTRANS = NONSEQ : read; state = read & HTRANS = NONSEQ & HWDATA : idle; 1 : error; esac; ... init (state) := idle; November 12, 2018

Functional Verification Domain-specific Model Outline Motivation Formal modeling Functionality Case study Performance Conclusion Model of an AMBA Slave Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations MODULE slave (HADDR, HTRANS, HWDATA, HRDATA, HREADY, HRESP, HMASTER, HSPLIT, MASK_MASTER1, MASK_MASTER2, MASK_MASTER3, SLAVE_STATE) VAR state : {idle, write, read, error}; prev state : {idle, write, read, error}; extended : boolean; ASSIGN init (state) := idle; init (prev_state) := state; init (extended) := 0; next (prev_state) := state; next (state) := case SLAVE_STATE != x : SLAVE_STATE; HRESP = SPLIT : idle; !HREADY : state; HTRANS = BUSY : state; HRESP = RETRY : prev state; state = idle & HTRANS = NONSEQ & HADDR : write; state = idle : state; state = write & HTRANS = NONSEQ : read; state = read & HTRANS = NONSEQ & HWDATA : idle; 1 : error; esac; ... next (state) := November 12, 2018

Functional Verification Domain-specific Model Outline Motivation Formal modeling Functionality Case study Performance Conclusion Model of an AMBA Slave Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations MODULE slave (HADDR, HTRANS, HWDATA, HRDATA, HREADY, HRESP, HMASTER, HSPLIT, MASK_MASTER1, MASK_MASTER2, MASK_MASTER3, SLAVE_STATE) VAR state : {idle, write, read, error}; prev state : {idle, write, read, error}; extended : boolean; ASSIGN init (state) := idle; init (prev_state) := state; init (extended) := 0; next (prev_state) := state; next (state) := case SLAVE_STATE != x : SLAVE_STATE; HRESP = SPLIT : idle; !HREADY : state; HTRANS = BUSY : state; HRESP = RETRY : prev state; state = idle & HTRANS = NONSEQ & HADDR : write; state = idle : state; state = write & HTRANS = NONSEQ : read; state = read & HTRANS = NONSEQ & HWDATA : idle; 1 : error; esac; ... SLAVE STATE != x : SLAVE STATE; November 12, 2018

Functional Verification Domain-specific Model Outline Motivation Formal modeling Functionality Case study Performance Conclusion Model of an AMBA Slave Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations MODULE slave (HADDR, HTRANS, HWDATA, HRDATA, HREADY, HRESP, HMASTER, HSPLIT, MASK_MASTER1, MASK_MASTER2, MASK_MASTER3, SLAVE_STATE) VAR state : {idle, write, read, error}; prev state : {idle, write, read, error}; extended : boolean; ASSIGN init (state) := idle; init (prev_state) := state; init (extended) := 0; next (prev_state) := state; next (state) := case SLAVE_STATE != x : SLAVE_STATE; HRESP = SPLIT : idle; !HREADY : state; HTRANS = BUSY : state; HRESP = RETRY : prev state; state = idle & HTRANS = NONSEQ & HADDR : write; state = idle : state; state = write & HTRANS = NONSEQ : read; state = read & HTRANS = NONSEQ & HWDATA : idle; 1 : error; esac; ... HRESP = SPLIT : idle; November 12, 2018

Functional Verification Outline Motivation Formal modeling Functionality Case study Performance Conclusion Functional Verification Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations The formal verification of the AMBA AHB protocol has been addressed by many researchers previously The formal models used for the analysis were manually extracted from the specification – the correctness of these models have been shown, not the protocol itself No inconsistencies found in the final version of the protocol Unspecified parts of the specification may have been manually resolved by the researchers Can unspecified/ambiguous parts of the specification cause problems? We have found an ambiguity in the specification that may lead to flawed implementations (deadlocks) November 12, 2018

Ambiguity in AMBA Specification Outline Motivation Formal modeling Functionality Case study Performance Conclusion Ambiguity in AMBA Specification Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations Slave can unmask master by issuing HSPLITx What happens if slave also asserts RETRY? The specification is ambiguous: “A slave which issues RETRY responses must only be accessed by one master at a time” Is splitting an access? Can a slave issue RETRY if it can split? HSPLITx may be lost in some implementations No acknowledgement Slave thinks master is unmasked Arbiter is waiting for unmask req. Masked Communication RETRY HSPLIT3 (unmask Master3) November 12, 2018

Functional Verification Domain-specific Model Outline Motivation Formal modeling Functionality Case study Performance Conclusion How to be Careful? Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations The specification warns that “both the SPLIT and RETRY transfer responses must be used with care to prevent bus deadlock” AMBA AHB does allow bus deadlocks in some cases It is up to the designer to ensure functional correctness How to be a “careful designer”? Most often evaluated using simulations – imperfect Experience – expensive Formal methods – time consuming, difficult Functionality of SoC designs needs to be verified Tools and modeling languages are needed that help in the application of functional verification to SoC designs November 12, 2018

Can we resolve the ambiguity? Outline Motivation Formal modeling Functionality Case study Performance Conclusion Can we resolve the ambiguity? Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations We have assumed that HRESP = RETRY and HSPLITx does not occur simultaneously We introduced formulas that evaluate to true infinitely often HREADY, HRESP = OK, HSPLIT = masterx Using these assumptions we could prove There are no deadlocks in the system – no states with no transitions enabled There are no livelocks in the system – no states from which only a subset of states is reachable that cannot provide the necessary functionality of the system November 12, 2018

Case Study – JPEG 2000 Encoder Outline Motivation Formal modeling Functionality Case study Performance Conclusion Case Study – JPEG 2000 Encoder Distributed compression method – works on tiles Convert tile pixels from unsigned integers to two’s complements RGB  YUV or RGB  YCbCr transformation DWT generates four sub-bands (computation int. – HW) Quantization when lossy compression is used EBCOT: Tier–1 (computation int. – HW), Tier–2 (control int. – CPU) November 12, 2018

Case Study – SoC Architecture Outline Motivation Formal modeling Functionality Case study Performance Conclusion Case Study – SoC Architecture November 12, 2018

Case Study – Simulation Framework Outline Motivation Formal modeling Functionality Case study Performance Conclusion Case Study – Simulation Framework Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations Implemented the model in SystemC Cycle-accurate at the transaction level, functional blocks are cycle-approximate Each component is declared as SC_MODULE Each SC_MODULE may have several threads (SC_THREAD) There are 42 threads altogether in our implementation – verification using the SystemC model is practically not feasible November 12, 2018

Functional Verification Domain-specific Model Outline Motivation Formal modeling Functionality Case study Performance Conclusion Simulation Results 1/2 Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations Simulation results for JPEG encoding using 64x64 pixel tiles. Scale: cycles Tier-1 Tier-2 DWT Image DWT ET Tier-1 BCET Tier-1 WCET Tier-2 ET Input Output End-to-end WC baboon 194 188 517 005 741 519 9 122 240 12 288 11 099 10 335 043 boat 165 141 737 046 8 750 875 10 046 10 044 857 goddesses 513 846 772 461 8 663 630 11 456 9 996 487 goldhill 242 055 747 954 8 672 436 10 376 9 978 464 lena 461 601 769 239 8 689 815 11 979 10 024 198 November 12, 2018

Functional Verification Domain-specific Model Outline Motivation Formal modeling Functionality Case study Performance Conclusion Simulation Results 2/2 Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations Simulation results for JPEG encoding using 128x128 pixel tiles. Scale: cycles Tier-1 Tier-2 DWT Image DWT ET Tier-1 BCET Tier-1 WCET Tier-2 ET Input Output End-to-end WC baboon 751 393 2 315 254 3 151 948 9 010 373 49 152 36 537 14 290 609 boat 1 764 568 3 086 892 8 758 372 41 719 13 990 027 goddesses 1 843 190 3 219 664 9 451 990 42 391 14 823 509 goldhill 2 325 098 3 173 076 8 768 459 41 645 14 090 307 lena 2 364 360 3 241 400 8 793 070 37 578 14 172 351 November 12, 2018

Model-based Performance Evaluation Outline Motivation Formal modeling Functionality Case study Performance Conclusion Model-based Performance Evaluation Property Verification Design feedback Simulation Model Functional Verification Perf. Evaluation Input Formal Model Semantic mapping Domain Specific Model AnalysisDomain Simulations Parameters Exhaustive state space search on the formal models using the annotations Abstract out best case and worst case timing information for each component November 12, 2018

Information from Simulations Outline Motivation Formal modeling Functionality Case study Performance Conclusion Information from Simulations [bcet, wcet] [bcet, wcet] [bcet, wcet] Accurate formal bus & communication model Size of messages sent through the bus [bcet, wcet] November 12, 2018

Formal Performance Evaluation Outline Motivation Formal modeling Functionality Case study Performance Conclusion Formal Performance Evaluation Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations The model checker performs an exhaustive state space search using the parameters obtained by simulations Computations are abstracted out, but the timing information used as parameters is as accurate as the simulations All the valid execution traces of the model are considered for the analysis, if a property is violated a counter-example can be obtained The analysis may be fully automated & driven by the domain-specific model November 12, 2018

Formal Performance Evaluation Outline Motivation Formal modeling Functionality Case study Performance Conclusion Formal Performance Evaluation Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations We use the formal models developed for the functional verification to evaluate the worst case end-to-end performance of the SoC We make several assumptions for performance analysis No communication errors, no unexpected delays Forced execution order using signals/interrupts Several restrictions are forced by the model checkers Yes/no questions, truncating parameters – decreased accuracy More flexible model checkers are needed for practical application Several manual changes in the FSM models Manually removed several (unused) transitions from the models Goal is to increase the model checking scalability November 12, 2018

Formal Performance Evaluation Outline Motivation Formal modeling Functionality Case study Performance Conclusion Formal Performance Evaluation Design feedback Simulation Model Functional Verification Perf. Evaluation Formal Model Semantic mapping Domain-specific Model Analysis Domain Simulations Worst case bounds on the end-to-end computation time of the digital camera SoC obtained using model checking. Scale: cycles Tile size WCET (by simulation) (by model checking) 64 × 64 pixel tiles 10 335 043 11 000 000 128 × 128 pixel tiles 14 823 509 17 000 000 November 12, 2018

Outline Motivation Formal modeling Functionality Case study Performance Conclusion We have presented a systematic formal method to evaluate AMBA-based SoC designs Formal model of AMBA AHB bus using SMV Functional verification method to find ambiguities from AHB specification which may cause flawed SoC designs, and propose a way to resolve these issues Apply formal methods for performance evaluation The proposed method can guarantee worst case end-to-end execution times of AMBA-based SoCs The described method may be fully automated Improvements are needed in model checkers to allow easier use and simulations as well November 12, 2018

Questions? The NuSMV AMBA models are available for download at http://alderis.ics.uci.edu/amba2 November 12, 2018