1. Automated Testing Environment
Concepts required for testing embedded systems adopted in this course
(quizzes, assignments and laboratories)
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

2. To be tackled today Why test, and what kinds of tests are there?
Difference between
TLD – Test Last Development
TDD – Test Driven Development (Test First Development)
How do you add tests?
More details on the testing process E-TDD and the testing toll E-UNIT
Other kinds of available tests – time and access
Design of custom TESTs and ASSERTS
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

3. Why test?
Unit test
I AM CURRENTLY BUILDING A NEW FUNCTION – does it work the way I expect?
Testing my and my partner’s understanding of the Design
WE ARE DESIGNING A NEW PRODUCT -- If we can’t explain (to ourselves via a test) the product ideas – how do we know when product is working?
Regression testing
MY PARTNER ADDED A NEW FEATURE to a product – how do I know it did not “break something” I wrote?
System testing
How do I PROVE TO THE CUSTOMER that the product (Lab. or assignment) works the way that was requested?
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

4. What are the problems with this sort of “Human intervention” test?
This code
is placed in Assignments1 directory
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

5. VisualDSP development environment (IDDE) – Communication overhead
Debugging Interface
“JTAG” boundary scan operation
Processor
Peripherals
VisualDSP++
program
Debugging Interface
HPPCIce
Lab. station (ICT318 / 320)
BF533 Evaluation Board
Each time we want to send a message (via printf( ) or cout) we must “stop” the processor from doing what we want it to do, and then send messages over the interface – Slow, plus the testing can impact on peripheral performance.
E.g. no sound while doing printing in Assignment 2 and Lab. 1
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

6. More issues with this sort of test!
Tests are mixed up “with production code”
Must remove tests before release to customer
Unreliable – may result in test messages at wrong time in released code
Difficult to “add” and “remove tests”
Can’t be automated – therefore unlikely to have tests run often during development – leads to unreliable product
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

7. E-TDD Tool E-TDD – embedded test driven development
Build-the-tests-first (during design) ideas for product development has been made popular by the “Agile” approach
Requires an automated testing approach (test framework)
In this class, we are going to use the E-Unit tool for its automated behaviour
Will use in both “test last” development (what most people currently do) and in “test first” development.
I (the customer) will provide tests so that you can check that you have developed the “right” stuff at the “high” level
You will build additional tests to check your own code at the “low level”
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

8. Test Environment
Can download “LabTestEnvironment.zip” file from the web (Check the September webpages for exact name of file)
LabX directory is where you keep your “project” code LabXcode_main.cpp LabX_start.cpp LabX_initializeASM.asm
LabXTests is where you keep the tests for LabX LabXtests_main.cpp (built by GUI) Tests_for_LabX_start.cpp Tests_for_LabX_initializeASM.cpp written in C++) E_UnitConnect.cpp (built by GUI)
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

9. Build a new VDSP project “Assignment1Test” and place in “AssignmentTests” directory
Now add certain standard files to the project
Same set of files for all assignments and laboratory
Use the automated features of the GUI to “convert” VDSP project to test project
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

10. Now add your Assignment1 subroutines files to the Assignment1Test project
Don’t move or copy the files into the test directory – just “add” them to the project
Note the use of “Assignment1.h” which has the prototypes for all “C++” and “assembly code” functions used (from Assignment dir.)
This files are “added” to the “Test” project and NOT copied from the Assignment1 directory
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

11. The GUI has automatically added the “Tests_main.cpp”
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

12. The GUI has automatically added a “TestGroup” file for you
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

13. Modify “Assignment1Tests.cpp” to perform the needed testing
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

14. Tests and Test Groups have a standard format
TESTGROUP NAME
TEST NAME
Add further tests at this location
TESTGROUP NAME,
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

15. Tests then need to be customized A lot of this can be automated by GUI
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

16. Test Components Remember format for “exams” and “quizzes”
The ACTUAL Test (This is a TEST MACRO)
CHECK(result1 == result2)
Would provided better test messages if we “self-documented the code”
int result1_CPP =
int result2_ASM =
CHECK(result_CPP == result_ASM);
Test CONTROL statements (ALL MACROS)
TEST_CONTROL( testgroup_name)
TEST_FILE_RUN_NOTIFICATION( testgroup_name)
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

17. Have added stub versions of functions
Note “code DEFECT”
There is NO return value placed in R0
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

18. Now build source files to make executable
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
Note the format of the error messages.

19. Fix the code. Review the code for syntax errors and “logical” errors
DON’T IGNORE WARNINGS
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

20. Now “build and run” the tests All the tests should fail (no code)
Quick test failure visualization
Detailed test failure information
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

21. Now “build and run” the tests All the tests should fail (no code)
Quick test failure visualization
Clicking on the “detailed test info” automatically takes you to the “test” that failed
Detailed test failure information
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

22. The test that failed
Clicking on the “detailed test info” automatically takes you to the “test” that failed
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

23. Test information modes are controlled in “Test_main”
Show Failures and Expected Failures only
Show Failures and Successes
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

24. Add further tests at this location
Add a separate test to show that result2 (from the ASM code) is equal to 42
Add further tests at this location
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

25. Gui automatically adds new test
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

26. Write the tests for “self-documenting” messages (What does “result1 == result2” mean if 200 tests present?)
HAVE
WANT
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

27. Write the “CPP” code to get to work but “Write the ASM for speed”
“../E-UNITIncludes/E-UNIT_Tests1Sept07.h”
FAIL
SUCCESS
SUCCESS
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

28. Get a “Volun-tell” to come to the front and write the “better” test
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

29. Available ASSERTS More can be “customized”
CHECK(expected == actual) – Success if passes
XF_CHECK(expected == actual) – Success if “fails”
CHECK_EQUAL(expected, actual)
XF_CHECK_EQUAL(expected, actual) – Success if fails
CHECK_CLOSE(expected, actual, tolerance)
CHECK_ARRAY_EQUAL(expected, actual, length)
CHECK_ARRAY_CLOSE(expected, actual, length, toler)
REPORT(“character message”);
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada

30. Handled today – some easy “non-programming” quiz and midterm questions
Why test, and what kinds of tests are there?
Difference between
TLD – Test Last Development
TDD – Test Driven Development (Test First Development)
How do you add tests for Assignment 1?
More details on the testing process E-TDD and the testing tool E-UNIT
Other kinds of available tests – time and access
Design of custom TESTs and ASSERTS
11/24/2018
Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada