Parasoft ADP solutions Automated Defects Prevention for Embedded Systems Software Development by Wiktor Grodowski

Mostly focuses on Functional Testing tied back to Requirements Design – Code – “Integration and Test” Early Testing is minimal and short-lived Testing efforts are focused on the isolation and subsequent fixing of “bugs” – Error Detection “Failure to Communicate” between Development, Test and Quality Assurance (QA) Observations on Software Testing.. Not much has changed in 25 years..

The Cost of Waiting

ADP = Automated Defects Prevention Defects Prevention != Error Detection ADP = Defects Prevention + Automation ADP is not a replacement for CMM / CMMI and alike. It is a change in a software development approach. ADP is practical, flexible, down to earth, based on over 20 years of experience What is ADP all about?

ADP originates from Deming's and Jurand's works Attempting to introduce Deming's methodology into SDLC Follow the basic steps: Identify a defect Find its cause Locate where and why it was introduced Implement preventive methods Monitor the process Start preventing errors instead of detecting them! A brief history

Deming's wheel  Plan – establish objectives and processes  Do – implement new process  Check – measure the results and compare against expected outcome  Act – analyze the differences, determine where to apply changes to include improvement

ADP principles  Principle 1: Establishment of Infrastructure  Build a strong foundation through integration of people and technology  Principle 2: Application of General Best Practices  Learn from others’ mistakes  MISRA, JSF, other coding standards  Principle 3: Customization of Best Practices  Learn from your own mistakes  Principle 4: Measurement and Tracking of Project Status  Understand the past and present to make decisions about the future  Principle 5: Automation  Let the computer do it!  Principle 6: Incremental Implementation of ADP’s Practices and Policies

Establishment of infrastructure  People – extending traditional roles  Must promote communication, efficiency, productivity and job satisfaction  Each role must include defects prevention tasks  Understanding roles  All must understand how to adhere to the roles they are given  Define group behavior  Technology – minimum and more  SCMS  Build system  BTS  Reporting system

Application of best practices  Most software projects share same characteristics  Best practices can be used to limit:  Human errors  Common software defects  Where do the best practices come from?  Different granularity of best practices:  Organizational-level (EVMS – Earned Value Management System, general guidelines for use of CMS)  Design-level (suggestions for usage of specific technologies or techniques, such as design patterns, Ajax or SOA)  Code-construction level (procedures for peer programming, writing unit tests, code review etc.)

Customization of best practices  Must address project-specific problems  Is based on the Deming's principles  Monitor process  Identify a defect  Find cause of a defect  Locate part of a process that let the defect slip through  Modify process to prevent the defect from reappearing  Monitor process (again)  Static analysis tool should be implemented to monitor both practices automatically

Measurement and tracking  Measures must be quantitative  Absolute values (number of defects)  Metrics (possession of a given attribute, confidence factor)  Measurement:  What and how can be measured?  What does the measure or metric indicate?  What decisions can be based on these information?  Tracking:  What data can be tracked?  How is the data tracked?  What do we gain from tracking this data?

Automation  What is automation?  „Activity that can be performed completely without human intervention”  “Automation of mundane and repetitive tasks improves people satisfaction and effectiveness”  Not to substitute people but to improve working conditions for people  Automation improves product quality  Automation facilitates human communication  Automation helps to implement and enforce best practices and organizational standards  Automation improves people productivity  Automation helps to collect measurement data

 Code reviews are time-intensive and human-intensive  Review process can be done with a “static analysis” tool  Quality focus is typically NOT built into product schedules so there is typically little time to do proper testing.  Automation is required to leverage these precious cycles  Developers write more testing procedures/code than the code they are developing for delivery!  ADP “automates” much of the driver, stubs and infrastructure creation as well as provide metrics, like code coverage  Developers typically do NOT like testing.  Some problems will never be solved! Why Automation is Important !

Incremental implementation of ADP  Continuous change through:  Learning  Adaptation  Innovation  Change, however, can be overwhelming  ADP needs to be implemented gradually, on a group-by-group basis  Pilot group(s)  Division(s)  Organization  Best practices should also be introduced one-by-one

Incremental implementation of ADP  ADP is both iterative and incremental – the phases do not have to follow sequentially

SC M Source Control code & tests TCM Team Configuration Manager Test Results Scheduled Build and Test Server Reporting Program Manager Architect Exemplary ADP Deployment Professional Editions Server Edition Architect Edition Nightly Results Developer Machines Architect / Technical lead Company Standards deployment Target device

 High costs of defects  Number of defects in a final product must be kept at minimal due to a high recall cost  Possibility of having human life or human safety at stake, dependent on a stability of an embedded device  Important time to market  Late product = lost money  Mandatory compliance  MISRA, JSF, DO-178B, others  Contractor's internal standards Relation to Embedded Systems

 Exemplary ROI  Unit testing caused drop in residual errors almost by a factor of 2. Is it worth it? Reve nues ($) ‏ Time (months)‏ On-time Delayed entry Peak revenue Peak revenue from delayed entry Market rise Market fall W2W Time D On-time Delayed Revenues ($)‏

Is it really worth it?  ADP  Non-ADP

 C++Test  Coding Standards analysis  Flow Analysis  Unit Testing - Coverage analysis  Code Review  Authoring and Automation  Concerto  Work Planning and Tracking  Reporting and Analytics  Acceptance testing  Data correlation  Process Visibility  Insure++  Runtime memory analysis Parasoft Tools supporting ADP

X Understands and anticipates common coding mistakes that can lead to poor software quality X Uses testing as a process measurement technique XX Uses testing results to fix errors X Learns from test results and improves processes and prevents future errors X Reduces probability of errors reoccurring X Addresses the critical roles in creating software X Provides tools and processes to keep team members code at same level X Uses testing results to fix the development process XX Find Errors ADPTestingBenefits Benefits of ADP

Questions & Answers