1. Software Process Control1 João Cangussu (CS) Department of Computer Science University of Texas at Dallas, TX Ray. A. DeCarlo College of Electrical and Computer Engineering Purdue University, West Lafayette, IN Scott Miller Aditya P. Mathur Department of Computer Sciences, Purdue University, West Lafayette, IN Monday September 27, 2004. University of Aeronautics and Astronautics. Beijing. China. Feedback Control in Software Process Management: Progress and Problems

2. Software Process Control2 Research Question Can we control the Software Development Process in a manner similar to how physical systems and processes are controlled ? The central problem in control is to find a technically feasible way to act on a given process so that the process adheres, as closely as possible to some desired behavior. Furthermore, this approximate behavior should be achieved in the face of uncertainty of the process and in the presence of uncontrollable external disturbances acting on the process. The fundamental control problem (Ref: Control System Design by G. C. Goodwin et al., Prentice Hall, 2001):

3. Software Process Control3 Research Methodology [1] 1.Understand how physical systems are controlled? 2. Understand how software systems relate to physical systems. Are there similarities? Are there differences? 3. Understand the theory and practice of the control of physical systems. 4.Can we borrow from this theory? If “yes,” then proceed further, else drink coffee or tea and think of another research direction. 5.Adapt control theory to the control of SDP and develop models and methods to control the SDP. 6.Study the behavior of the models and methods in commercial settings.

4. Software Process Control4 Research Methodology [2] 7.Improve the model and methods. 8.Repeat steps 6 and 7 until you are thoroughly bored or get rich.

5. Software Process Control5 Feedback Control Specifications Program Effort + f(e) Additional effort What is f ? - Required Quality Observed Quality

6. Software Process Control6 Software Development Process: Definitions A Software Development Process (SDP) is a sequence of well defined activities used in the production of software. An SDP usually consists of several sub-processes that may or may not operate in a sequence. The Design Process, the Software Test Process, and the Configuration Management Process are examples of sub-processes of the SDP.

7. Software Process Control7 Software Development Process: A Life Cycle Requirements Elicitation Requirements Analysis Integrate/Test Design Code/Unit test System test More testDeploy Not all feedback loops are shown.

8. Software Process Control8 Initial Focus Software Test Process (STP): System test phase Objective: Control the STP so that the quality of the tested software is as desired. Quantification of quality of software: Number of remaining errors Reliability

9. Software Process Control9 Current Focus Actual Process Parameter Estimation Progress Metrics Process Model Parameters - + Feedback Controller Estimated Future Schedule Deviation Estimation Error Predicted Schedule   - + Suggested Decision Changes Mgmt. Control Management Decisions Desired Schedule Actual Schedule

10. Software Process Control10 Problem Scenario cp 1 cp 2 cp 3 cp 4 cp 5 cp 6 cp 7 cp 8 cp 9 cp i = check point i rfrf schedule set by the manager approximation r0r0 observed deadline r - number of remaining errors t- time t0t0

11. Software Process Control11 Initial Approach Actual STP Controller r error (t)  ’  w’ f + + wf+wfwf+wf  +  wf+wfwf+wf STP State Model r observed (t) r expected (t) scsc r0r0 scsc r0r0 Initial Settings (w f,  )  wfwf Test Manager

12. Software Process Control12 Physical and Software Systems: An Analogy Dashpot Rigid surface External force Xcurrent Xequilibrium X: Position Number of remaining errors Spring Force Effective Test Effort Block Software Mass of the block Software complexity Quality of the test process Viscosity Spring To err is Human

13. Software Process Control13 Physical Systems: Control Controllability Is it possible to control X (r) by adjusting Y (workforce and process quality)? Observability Does the system have distinct states that cannot be unambiguously identified by the controller ? Robustness Will control be regained satisfactorily after an unexpected disturbance?

14. Software Process Control14 Physical Systems: Laws of Motion [1] First Law: Every object in a state of uniform motion tends to remain in that state of motion unless an external force is applied to it. Does not apply to testing because the number of errors does not change when no external effort is applied to the application.

15. Software Process Control15 Physical Systems: Laws of Motion [2] Newton’s Second Law: The relationship between an object's mass m, its acceleration a, and the applied force F is F = ma. CDM’s First Law: The relationship between the complexity Sc of an application, its rate of reduction in the number of remaining errors, and the applied effort E is E=Sc. r..

16. Software Process Control16 Physical Systems: Laws of Motion [3] Third Law: For every action force, there is an equal and opposite reaction force. When an effort is applied to test software, it leads to fatigue on the tester. This is not a scientific relationship.

17. Software Process Control17 Assumption I: CDM’s First Law The magnitude of the rate of decrease of the remaining errors is directly proportional to the net applied effort and inversely proportional to the complexity of the program under test. This is analogous to Newton’s Second Law of motion. r.. = E/S c  r.. E= S c

18. Software Process Control18 Assumption II: CDM’s Second Law The magnitude of the effective test effort is proportional to the product of the applied work force and number of remaining errors. Analogy with the spring: for an appropriate .

19. Software Process Control19 Assumption III: CDM’s Third Law The error reduction resistance is proportional to the error reduction velocity and inversely proportional to the overall quality of the test phase. Analogy with the dashpot: for an appropriate . r. erer  1/  =

20. Software Process Control20 State Model : Disturbance

21. Software Process Control21 Computing the feedback

22. Software Process Control22 Case Study II: Razorfish Project Description Project Goal: translate 4 million lines of Cobol code to SAP/R3 A tool has been developed to achieve the goal of this project. Goal of the test process: (a) Test the generated code, not the tool. (b) Reduce the number of errors by about 85%.

23. Software Process Control23 Validation: Razorfish Project Testing Process Transformer = modify S SAP R/3 run output 1 output 2 run S Cobol Select a Test Profile input continue testing yes no

24. Software Process Control24 Case Study II: Razorfish Project Results 85% reduction achieved. If the process parameters are not altered then the goal is reached in about 35 weeks.

25. Software Process Control25 Alternatives from Feedback: STP Quality Desired eigenvalue=-0.152 Improving quality alone will not help in achieving the goal.

26. Software Process Control26 Alternatives from Feedback: Workforce Desired eigenvalue=-0.152 Changing the workforce alone can produce the desired results.

27. Software Process Control27 Alternatives from Feedback: STP quality and workforce Set of valid choices for changing the quality and the workforce

28. Software Process Control28 Summary Analogy between physical and software systems presented. The notion of feedback control of software processes introduced. One case study described. Parameter estimation techniques used for model calibration. Made use of system identification techniques.

29. Software Process Control29 Issues 1.Difficult to conduct case studies where the control suggested by the model is actually applied. 2.Model of the incremental test process that includes links to the development process is non-linear. Controller yet to be developed. 3.There is too much variation in the implementation of the test process. This needs to be modeled using, perhaps, noise.