Ray. A. DeCarlo School of Electrical and Computer Engineering Purdue University, West Lafayette, IN Aditya P. Mathur Department of Computer Science Friday September 30, COMPSAC Hong Kong. Modeling and Controlling the Incremental Software Test Process Scott Miller (CS) Department of Computer Science

The Problem What actions must the management take in order to adhere to process schedule?

Approaches 1.Use instinct and experience. 2.Use (1) supported by quantitative tools. (a)Use simulation: “forward” approach (b)Use (a) plus feedback control: “inverse” approach. Our approach..

Decision Support via Feedback Actual Process Parameter Estimation Progress Metrics Process Model Parameters Feedback Controller Estimated Future Schedule Deviation Suggested Decision Changes Mgmt. Control Management Decisions - Predicted Schedule   - + Desired Schedule Actual Schedule + Estimation Error

A Flow Model of Incremental Software Development/Test Test Specs Test Authoring Feat. Specs Test Code Coding Test Verification & Correction Reg. Cases Regression Testing Known Defects Code Debugging Project Code Latent Defects

Workforce Allocation Workforce allocated to particular tasks Effort is split across all active tasks

State-Model [Example] Equations System State Progress Feature Coding (fc) Code Debugging (dr) Test Authoring (ta) Test Debugging (td) Regression Testing (rr) Defect Model Development Testing

Variable Productivity Equation Human Productivity Workload Dependent (Csikszentmihalyi,’88) r b – Base Work rate c – Fractional size- dependent increase w c – Current workload size w n – Nominal workload size

The “Productivity” Eqns. Process Productivity (E.g. Feature Coding) Defect Introduction Defect Detection (Cangussu et al., ’02)

Control Strategy Model Predictive Control

Select Cost Functional E.g. Q 1,Q 2 := positive definite Calculate where S[x k, u k,k+P ]  x p k,k+P

Initial Study Data Collection Questionnaire Unavailable Data Estimated by Mgmt. Many “Linear Approximations” Many subjective estimates Results follow

Scheduled Simulated Expected Coding Progress Features Rel. 1 Features Rel. 2 Features Rel. 3 Inferred Actual

Expected Test Authoring Progress Scheduled Simulated Tests Rel. 1 Tests Rel. 2 Tests Rel. 3 Actual

Expected Test Debugging Progress Scheduled Actual Simulated Tests Rel. 1 Tests Rel. 2 Tests Rel. 3

Expected Regression Progress Scheduled Inferred Actual Simulated Regression Tests Rel. 1 Regression Tests Rel. 2 Regression Tests Rel. 3

Ongoing Study Data collection tool Objective parameter estimation Will reuse the tailored flow model Goal: Assess predictive accuracy Goal: Observe predictive accuracy over evolving process (i.e. training data growth)