University of Southern California Center for Software Engineering C S E USC ISERN 2005 November 15, 2005 Stefan Biffl, Aybuke Aurum, Rick Selby, Dan Port, Barry Boehm Value-Based Empirical Methods (VBEM) Session

University of Southern California Center for Software Engineering C S E USC 11/15/03©USC-CSE2 Summary VBEM’s support better use of scarce resources –Focus on high-value aspects –Some ISESE 2005 results VBEM’s present challenges –Values vary by situation, time –Mix of qualitative and quantitative data –Data often competitively sensitive Speakers provide 10-minute experience summaries –Major opportunity areas, problem areas –Followed by discussion of these

University of Southern California Center for Software Engineering C S E USC 11/15/03©USC-CSE3 Value-Based Testing: Empirical Data and ROI — LiGuo Huang, ISESE 2005 (a) (b)

University of Southern California Center for Software Engineering C S E USC 11/15/03©USC-CSE4 COCOMO II: Added % test time COQUALMO: P q (L) Value-Based: S q (L): Pareto Value-Neutral: S q (L): Linear Market Risk: RE m Sweet Spot Value/Risk-Driven Testing: 40% Gain — LiGuo Huang, ISESE 2005

University of Southern California Center for Software Engineering C S E USC 11/15/03©USC-CSE5 By NumberP-value% Gr A higherBy ImpactP-value% Gr A higher Average of Concerns Average Impact of Concerns Average of Problems Average Impact of Problems Average of Concerns per hour Average Cost Effectiveness of Concerns Average of Problems per hour Average Cost Effectiveness of Problems Group A: 15 IV&V personnel using VBR procedures and checklists Group B 13 IV&V personnel using previous value-neutral checklists – Significantly higher numbers of trivial typo and grammar faults Experiment Value-Based Reading (VBR) Experiment — Keun Lee, ISESE 2005