Test Abstractions Intent Nat. Lang. Spec. HW Behavioral Tests can be described at different abstraction levels Tests can be defined top-down or bottom-up Example: A Blackjack Machine Check if the computer obeys the dealer rules when it has 17 or more points Player: Immediately stands. Dealer should stand when dscore is >= 17 pStand <= 0; (dHit) if (dscore > 17) $display(“error”);

Black Box Testing Testing without knowledge of the internals of the design. Only know system inputs/outputs Testing the functionality without considering implementation Inherently top-down 1.Test a multiplier by supplying random numbers to multiply 2.Test an anti-lock braking system by hitting the brakes at different speeds 3.Test a traffic light controller by simulating pre-recorded traffic patterns Black Box Testing Examples:

Black Box Testing Issues Advantages: Tests can be generated early in the design process Tests can often be reused after design change Disadvantages: Less information is available for test generation More manual process because no simulatable info exists Example: Random test generation

Specification-Based Test A block box testing approach Natural language specification is used to generate test sequence Goal is to generate tests which verify all aspects of the specification Extract a featureCreate nat. lang testsCreate executable tests Until all features have tests Clear relationship between features and tests Good to accommodate specification change

Extracting Features from a Specification Purely manual process Specs are long and vague The dealer must continue to take cards ("hit") until his total is 17 or greater. An Ace in the dealer's hand is always counted as 11 if possible without the dealer going over 21. Part of a Blackjack Specification Each sentence could be a feature for testing

A Feature from Blackjack A blackjack, or natural, is a total of 21 in your first two cards. If you split a pair of Aces for example, and then draw a ten-valued card on one of the Aces, this is not a blackjack, but rather a total of 21. The distinction is important, because a winning blackjack pays the player odds of 3 to 2. This detail is exactly something that a designer might miss Maybe the designer doesn’t appreciate the difference (3 to 2 odds)

Another Feature from Blackjack If the dealer turns an up-card of an Ace, he will offer "Insurance" to the players. Insurance bets can be made by betting up to half your original bet amount. The dealer will check to see if he has a 10-value card underneath his Ace, and if he does have Blackjack, your winning Insurance bet will be paid at odds of 2:1. Do you know what an “up-card” is? Is it clear that a player making an insurance bet ends up breaking even if the dealer has blackjack? Gamblers understand this, designers may not. Specifications often assume the background of the reader

Natural Language Test Descriptions  For each feature, describe tests to validate each feature Requirements of a Test Description 1.Describe sequence at controllable points Variables/signals which can be controlled during testing (inputs or internal) 2.Describe sequence at observable points Variables/signals where results are known (outputs or internal) 3.Describe timing if relevant to the application

Controllable and Observable Points Need to select variable/signals to apply test data and to observe test results Need minimal structural detail (I/O, etc.) Controllable points - Inputs are the obvious choice Other points are chosen to reduce test time or control randomness  Setting an internal 8 bit counter value  Bypassing a random process Observable points - Output are the obvious points Other points are chosen to reduce test time and localize errors  Observe internal regs in a microprocessor

Sample Test Description The dealer must continue to take cards ("hit") until his total is 17 or greater. The player’s choices should not matter Will any test validate this feature? No. - dealer has 16 and draws a 6 Need to control the “random” deal Possible test sequence: Player stands, dealer receives 10 and 7.

Sample Test Description Possible test sequence: Player bets X dollars. Player hits and draws an ace. Player hits and draws a 10. Verify 3 to 2 payoff. A blackjack, or natural, is a total of 21 in your first two cards. If you split a pair of Aces for example, and then draw a ten-valued card on one of the Aces, this is not a blackjack, but rather a total of 21. The distinction is important, because a winning blackjack pays the player odds of 3 to 2. Test this by causing the player to make a bet, draw a blackjack, and check the winnings

Create Executable Tests Write the verilog test bench to implement each test Need to know all controllable and observable points in detail Test sequence: Player stands, dealer receives 10 and 7. module BlackJack (player_choice, dealt_card, dealer_choice, …) player_choice = 1; // stand If (dealer_choice != 0) $display (“error”); #1 dealt_card = 10; If (dealer_choice != 0) $display(“error”); #1 dealt_card = 7; If (dealer_choice != 1) $display (“error”);