1. Parasoft ADP solutions Automated Defects Prevention for Embedded Systems Software Development by Wiktor Grodowski

2. Mostly focuses on Functional Testing tied back to Requirements Design – Code – “Integration and Test” Early Testing is minimal and short-lived Testing efforts are focused on the isolation and subsequent fixing of “bugs” – Error Detection “Failure to Communicate” between Development, Test and Quality Assurance (QA) Observations on Software Testing.. Not much has changed in 25 years..

3. The Cost of Waiting

4. ADP = Automated Defects Prevention Defects Prevention != Error Detection ADP = Defects Prevention + Automation ADP is not a replacement for CMM / CMMI and alike. It is a change in a software development approach. ADP is practical, flexible, down to earth, based on over 20 years of experience http://www.adpqb.org What is ADP all about?

5. ADP originates from Deming's and Jurand's works Attempting to introduce Deming's methodology into SDLC Follow the basic steps: Identify a defect Find its cause Locate where and why it was introduced Implement preventive methods Monitor the process Start preventing errors instead of detecting them! A brief history

6. Deming's wheel  Plan – establish objectives and processes  Do – implement new process  Check – measure the results and compare against expected outcome  Act – analyze the differences, determine where to apply changes to include improvement

7. ADP principles  Principle 1: Establishment of Infrastructure  Build a strong foundation through integration of people and technology  Principle 2: Application of General Best Practices  Learn from others’ mistakes  MISRA, JSF, other coding standards  Principle 3: Customization of Best Practices  Learn from your own mistakes  Principle 4: Measurement and Tracking of Project Status  Understand the past and present to make decisions about the future  Principle 5: Automation  Let the computer do it!  Principle 6: Incremental Implementation of ADP’s Practices and Policies

8. Establishment of infrastructure  People – extending traditional roles  Must promote communication, efficiency, productivity and job satisfaction  Each role must include defects prevention tasks  Understanding roles  All must understand how to adhere to the roles they are given  Define group behavior  Technology – minimum and more  SCMS  Build system  BTS  Reporting system

9. Application of best practices  Most software projects share same characteristics  Best practices can be used to limit:  Human errors  Common software defects  Where do the best practices come from?  Different granularity of best practices:  Organizational-level (EVMS – Earned Value Management System, general guidelines for use of CMS)  Design-level (suggestions for usage of specific technologies or techniques, such as design patterns, Ajax or SOA)  Code-construction level (procedures for peer programming, writing unit tests, code review etc.)

10. Customization of best practices  Must address project-specific problems  Is based on the Deming's principles  Monitor process  Identify a defect  Find cause of a defect  Locate part of a process that let the defect slip through  Modify process to prevent the defect from reappearing  Monitor process (again)  Static analysis tool should be implemented to monitor both practices automatically

11. Measurement and tracking  Measures must be quantitative  Absolute values (number of defects)  Metrics (possession of a given attribute, confidence factor)  Measurement:  What and how can be measured?  What does the measure or metric indicate?  What decisions can be based on these information?  Tracking:  What data can be tracked?  How is the data tracked?  What do we gain from tracking this data?

12. Automation  What is automation?  „Activity that can be performed completely without human intervention”  “Automation of mundane and repetitive tasks improves people satisfaction and effectiveness”  Not to substitute people but to improve working conditions for people  Automation improves product quality  Automation facilitates human communication  Automation helps to implement and enforce best practices and organizational standards  Automation improves people productivity  Automation helps to collect measurement data

13.  Code reviews are time-intensive and human-intensive  Review process can be done with a “static analysis” tool  Quality focus is typically NOT built into product schedules so there is typically little time to do proper testing.  Automation is required to leverage these precious cycles  Developers write more testing procedures/code than the code they are developing for delivery!  ADP “automates” much of the driver, stubs and infrastructure creation as well as provide metrics, like code coverage  Developers typically do NOT like testing.  Some problems will never be solved! Why Automation is Important !

14. Incremental implementation of ADP  Continuous change through:  Learning  Adaptation  Innovation  Change, however, can be overwhelming  ADP needs to be implemented gradually, on a group-by-group basis  Pilot group(s)  Division(s)  Organization  Best practices should also be introduced one-by-one

15. Incremental implementation of ADP  ADP is both iterative and incremental – the phases do not have to follow sequentially

16. SC M Source Control code & tests TCM Team Configuration Manager Test Results Scheduled Build and Test Server Reporting Program Manager Architect Exemplary ADP Deployment Professional Editions Server Edition Architect Edition Nightly Results Developer Machines Architect / Technical lead Company Standards deployment Target device

17.  High costs of defects  Number of defects in a final product must be kept at minimal due to a high recall cost  Possibility of having human life or human safety at stake, dependent on a stability of an embedded device  Important time to market  Late product = lost money  Mandatory compliance  MISRA, JSF, DO-178B, others  Contractor's internal standards Relation to Embedded Systems

18.  Exemplary ROI  Unit testing caused drop in residual errors almost by a factor of 2. Is it worth it? Reve nues ($) ‏ Time (months)‏ On-time Delayed entry Peak revenue Peak revenue from delayed entry Market rise Market fall W2W Time D On-time Delayed Revenues ($)‏

19. Is it really worth it?  ADP  Non-ADP

20.  C++Test  Coding Standards analysis  Flow Analysis  Unit Testing - Coverage analysis  Code Review  Authoring and Automation  Concerto  Work Planning and Tracking  Reporting and Analytics  Acceptance testing  Data correlation  Process Visibility  Insure++  Runtime memory analysis Parasoft Tools supporting ADP

21. X Understands and anticipates common coding mistakes that can lead to poor software quality X Uses testing as a process measurement technique XX Uses testing results to fix errors X Learns from test results and improves processes and prevents future errors X Reduces probability of errors reoccurring X Addresses the critical roles in creating software X Provides tools and processes to keep team members code at same level X Uses testing results to fix the development process XX Find Errors ADPTestingBenefits Benefits of ADP

22. Questions & Answers www.parasoft-embedded.com