1. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 1 Automated Testing Environment Concepts required for testing embedded systems adopted in this course (quizzes, assignments and laboratories)

2. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 2 / 28 To be tackled today Why test, and what kinds of tests are there? Difference between TLD – Test Last Development TDD – Test Driven Development (Test First Development) How do you add tests? More details on the testing process Other kinds of available tests – time and access Design of custom TESTs and ASSERTS

3. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 3 / 28 Why test? Unit test I AM CURRENTLY BUILDING A NEW FUNCTION – does it work the way I expect? Testing my and my partner’s understanding of the Design WE ARE DESIGNING A NEW PRODUCT -- If we can’t explain (to ourselves via a test) the product ideas – how do we know when product is working? Regression testing MY PARTNER ADDED A NEW FEATURE to a product – how do I know it did not “break something” I wrote? System testing PROOF TO THE CUSTOMER that the product (Lab. or assignment) works the way that was requested.

4. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 4 / 28 What are the problems with this sort of test?

5. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 5 / 28 VisualDSP development environment (IDDE) Debugging Interface Processor Peripherals BF533 Evaluation Board VisualDSP++ program Debugging Interface HPPCIce Lab. station (ICT318 / 320) Each time we want to send a message (via printf( ) or cout) we must “stop” the processor from doing what we want it to do, and then send messages over the interface – Slow, plus the testing can impact on peripheral performance. E.g. no sound while printing in Assignment 2 and Lab. 1

6. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 6 / 28 More issues with this sort of test! Tests are mixed up “with production code” Must remove tests before release to customer Unreliable – may result in test messages at wrong time Difficult to “add” and “remove tests” Can’t be automated – therefore unlikely to have tests run often during development – leads to unreliable product

7. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 7 / 28 E-TDD Tool E-TDD – embedded test driven development Build-the-tests-first ideas for product development has been made popular by the “Agile” approach Requires an automated testing approach In this class, we are going to use the E-TDD tool for its automated behaviour Will use in both “test last” development (what most people currently do) and in “test first” development I (customer) will provide tests so that you can check that you have developed the “right” stuff at the “high” level You will build additional tests to check your own code at the “low level”

8. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 8 / 28 Test Environment Can download “LabTestEnvironment.zip” file from the web LabX directory is where you keep your code LabXTests is where you keep the tests for LabX E-TDDIncludes directory contains “everything” needed to make test environment work

9. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 9 / 28 Build a new VDSP project “Assignment1Test” and place in “AssignmentTests” directory Now add certain standard files to the project – same set of files for all assignments and laboratory All found in E-TDDIncludes directory

10. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 10 / 28 Now add your Assignment1 subroutines files to the Assignment1Test project Don’t move or copy the files into the test directory – just “add” them to the project Note the use of “Assignment1.h” which has the prototypes for all “C++” and “assembly code” functions used (from Assignment dir.) This files are “added” NOT copied from the Assignment1 directory

11. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 11 / 28 Now we need to develop and add “Testmain.cpp”

12. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 12 / 28 Now we need to develop and add Assignment1Tests.cpp” 123456789123456789

13. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 13 / 28 Tests have a standard format 1348913489 TESTGROUP NAME TESTGROUP NAME,TEST TYPE TEST NAME Add further tests at this location

14. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 14 / 28 Tests then need to be customized 123456789123456789

15. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 15 / 28 Test Components The ACTUAL Test (This is a TEST MACRO) CHECK(result1 == result2) Would provided better test messages if we “self-documented the code” CHECK(result_CPP == result_ASM); Test CONTROL statements (ALL MACROS) TEST_CONNECT( test_name) LINK_TEST( test_name, test_type) TEST_LEVEL( which_test_level)

16. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 16 / 28 Stages to “automating” the tests 1)Activate the E-TDD Gui (E-TDDIncludes directory) This causes the GUI to appear, and adds new “menus” to VisualDSP

17. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 17 / 28 Stages to “automating” the tests 2) Select TDD Connection | Refresh Connection List This automatically builds the Force Connect Test file

18. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 18 / 28 Now “build and run” the tests Quick test failure visualization Detailed test failure information Clicking on the “detailed test info” automatically takes you to the “test” that failed

19. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 19 / 28 Test information modes are controlled via “ActivateTests.cpp” Show Failures only Show Failures and Successes

20. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 20 / 28 Add a separate test to show that result2 (from the ASM code) is equal to 7 Add further tests at this location

21. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 21 / 28 Write the tests for “self-documenting” messages (What does “result1 == result2” mean if 200 tests present?) HAVE WANT

22. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 22 / 28 Write the “CPP” code to get to work but “Write the ASM for speed” FAIL SUCCESS

23. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 23 / 28 Did the ASM code “destroy” or even “use” the CPParray[ ] defined in CPPassignment1.cpp?

24. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 24 / 28 Available ASSERTS LONG INT tests – 32 bit – typical for “C++” CHECK(condition) – Success if passes CHECK_EQUAL(expected, actual) EXPECTED_FALSE(condition) – Success if fails ARRAYS_EQUAL(expected, actual, length) CHECK_VALUES(expectedTemp, condition, actualTemp) DOUBLES_EQUAL(expected, actual, threshold)

25. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 25 / 28 ASSERTS for embedded systems Short int tests – 16 bit – typical for embedded systems SHORTS_EQUAL(expected, actual) SHORTS_CHECK(expected, condition, actual) USHORTS_EQUAL(expected, actual) USHORTS_CHECK(expected, condition, actual) Timing of routine – Blackfin Specific MAXTIME_ASSERT(max_time, function) Useful utility MEASURE_EXECUTION_TIME(timetaken, function) Examples available in the tests associated with Lab. 1

26. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 26 / 28 You can add your own ASSERTS ASSERT FORMAT #define CHECK_VALUES(expectedTemp, condition, actualTemp)\ if (!((expectedTemp) condition (actualTemp))) \ {\ FAILURE1(#expectedTemp##" !"#condition##" "#actualTemp);\ }\ else {\ SUCCESS1(#expectedTemp##”""#condition##" "#actualTemp);\ }

27. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 27 / 28 Practice – Assert Design Design an assert that will determine whether the lower 8 bits of a 32-bit variable is identical to the lower 8 bits of a second 32- bitvariable. The two variables should remain unchanged by the assert (since they may be reused later) The upper 24 bits of each variable MUST be ignored in the assert Does it matter if the variables were defined in the.data section of an assembly code routine or in an C++ code routine?

28. 6/2/2015 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada 28 / 28 Handled today – some easy quiz and midterm questions Why test, and what kinds of tests are there? Difference between TLD – Test Last Development TDD – Test Driven Development (Test First Development) How do you add tests for Assignment 1? More details on the testing process Other kinds of available tests – time and access Design of custom TESTs and ASSERTS