1 FIT for Embedded System Test Development PRESENTER : Jingwen Chen

2 Agile methodology from the desktop world XP Inspired embedded system life-cycle Adapting FIT into embedded system development Adapting FIT into embedded system development –What is FIT/FitNesse –EmbeddedFitNesse testing framework –MatLab/C++ FitNesse testing framework Issues with FIT in embedded design Issues with FIT in embedded design Conclusion Conclusion Main topics

3 What we want to avoid ? System defects System defects –Software errors cost $59 billion / year –Baxter Healthcare has to recall all its 256,000 Colleague Volumetric Infusion Pumps because the hardware / software problems that already cause 3 deaths and 6 serious injuries. We want to find an appropriate and efficient defect free development process for embedded devices We want to find an appropriate and efficient defect free development process for embedded devices

4 Agile methodology from the desktop business world Minimize risk by developing software in short iterations, ech iteration is like a miniature software project of its own Minimize risk by developing software in short iterations, ech iteration is like a miniature software project of its own Extreme Programming (XP) project.html Extreme Programming (XP) project.html project.html project.html

5

6 Agile methodology from the desktop business world Explore design space Explore design space –Test Driven Development (TDD) Improved defect reduction Improved defect reduction Enhanced communication between the customers and developers Enhanced communication between the customers and developers –Extremes of knowledge within a multi-disciplinary team –Customer: product end-users and researchers –Developer: Engineers

7 XP inspired (XPI) product lifecycle for biomedical embedded system Stage 1 : XPI Product envisagement –Customers together with developers produce “executable system specification” –System acceptance tests Stage 2 : XPI Proof of concept –Algorithms developed in MATLAB and validated by the acceptance tests written in first stage –Additional unit test in MATLAB using MUnit

8 XP inspired (XPI) product lifecycle for biomedical embedded system Stage 3 : XPI Prototyping of initial product –Forming the initial product using host machine simulation / embedded system –System moved into the C/C++ production language and validated via tests written in stage 1 and stage 2 –Additional embedded unit tests added for new environment. Stage 4 : XPI Full production of system –The production of the full system on real embedded system target board –Satisfy the acceptance tests and unit tests developed

9 Build an Agile Build an appropriate Agile development environment for embedded system A mechanism to allow customers without embedded system knowledge to form a part of the development team A mechanism to allow customers without embedded system knowledge to form a part of the development team FitNesse/FIT is one possible solution FitNesse/FIT is one possible solution

10 What is FIT/FitNesse ? A software testing tool designed for customers with limited IT knowledge A wiki-based system for creating, organizing, and running FIT test tables A wiki-based system for creating, organizing, and running FIT test tables Fit test tables are easy to be read and written by customer

11 An example of FIT test table TestFixtures. CheckLifeSign() itemsinputValuetestResults? bloodPressure95normal heartRate80normal temperature38 high expected normal actual

12 Starting from a simple temperature measurement embedded system Test software function: CalculateTemperature ( ) running on the target board convert pulse width to actual temperature Target Platform Hardware Layer Analog Device ADSP-BF533 processor System Software Layer Application Layer CalculateTemperature( ) VDK real-time operating system Input device: TMP03 thermal sensor Voltage pulse: Time Low Time High

13 Stage 1. XPI Product envisagement Customers together with the developers write the Fit test table Customers together with the developers write the Fit test table DSPFixture.VDSPColumnFixtureTest VoltHigh VoltLow outputTemperature(F)

14 Stage 2. XP Inspired Proof of concepts MatLabFitNesse testing framework MatLabFitNesse testing framework Developing the function in MatLab validated by the fit test in stage 1 Developing the function in MatLab validated by the fit test in stage 1 Fixture for Test table FitNesse Web server Wiki editor and storage DSPFixture.VDSPColumnFixtureTest VoltHigh VoltLowoutputTemperature(F) expected 135 actual MATLAB ENGINE MATLAB API Runner Fit Server

15 Migrate the MatLab code into embedded system language validated via tests for stage 1 and stage 2 using host machine simulation Migrate the MatLab code into embedded system language validated via tests for stage 1 and stage 2 using host machine simulation HOST MACHINE EmbeddedFitNesse Fixture for Test table Runner Fit Server C++ Simulation Environment Stage 3. XPI initial production system FitNesse Web server Wiki editor and storage DSPFixture.VDSPColumnFixtureTest VoltHigh VoltLowoutputTemperature(F) expected 135 actual MATLAB ENGINE MATLAB API

16 Running the function on real embedded target platform and satisfying the original test in stage 1 Running the function on real embedded target platform and satisfying the original test in stage 1 HOST MACHINE EmbeddedFitNesse Fixture for Test table DSP Interface class S.M.I.L.E TARGET PLATFORM Stage 4. XPI full production system MATLAB ENGINE MATLAB API FitNesse Web server Wiki editor and storage DSPFixture.VDSPColumnFixtureTest VoltHigh VoltLowoutputTemperature(F) expected 135 actual VDSP API VDSP Environment Runner Fit Server C++ Simulation Environment

17 Some tests we have done for embedded hardware: Lab2TestFixtures.SetCoreTimerAssembly setCOUNTsetPERIODsetSCALEreadTCOUNT()readTPERIOD()readTSCALE() expected actual An error on the ADSP-BF533 processor was found by undergraduate students taking our embedded systems course. They were using TDD to develop the register setting function SetCoreTimer ( ) running on the embedded target board.

18 HOST MACHINE EmbeddedFitNesse Fixture for Test table Issues with the full EmbeddedFitNesse testing framework FitNesse Web server Wiki editor and storage Long communication path ! S.M.I.L.E TARGET PLATFORM DSPFixture.VDSPColumnFixtureTest VoltHigh VoltLowoutputTemperature(F) expected 135 actual DSP Interface class VDSP API VDSP Environment Runner Fit Server C++ Simulation Environment MATLAB ENGINE MATLAB API

19 FitNesse in embedded design How to use FIT tables to specify an embedded system properly ? – –Embedded system acceptance tests includes: system operations external and internal stimulus responses interrupt latency system reliability – –Extend the fit test table to handle diverse test cases. timing diagrams signal comparison figures Efficiency and reliability concerns – –Make the EmbeddedFitNesse testing frame work more efficiently Is FitNesse a good solution for embedded system Agile development ?

20 XPI Lifecycle embedded system development EmbeddedFitNesse testing framework EmbeddedFitNesse testing framework MatLab/C++ FitNesse testing framework MatLab/C++ FitNesse testing framework Issues with FIT in embedded design Issues with FIT in embedded design Conclusion

21 Thank you !