2/13/07(c) USC-CSSE1 An Empirical Study on MBASE and LeanMBASE Supannika Koolmanojwong Center for Systems and Software Engineering CSSE- Annual Research Review February 13, 2007

2/13/07(c) USC-CSSE2 Outline Introduction Research Questions MBASE & its problem LeanMBASE –Content Comparison –Performance Comparison Conclusion Future Work

2/13/07(c) USC-CSSE3 Introduction Light weighted Software Engineering Projects –CSCI577ab: Software Engineering Course Sequence for graduate students in Computer Science –12 weeks in Fall semester, 12 weeks in Spring semester –6 on-campus students, 2 off-campus students –From , use MBASE as methodology to develop real-client projects. Problems –MBASE is too heavy for software engineering classes.

2/13/07(c) USC-CSSE4 Research Question With small-sized, limited schedule, real client project, how can we provide the best opportunity for students to learn good software engineering approaches and apply them to the real software project development?

2/13/07(c) USC-CSSE5 MBASE Model-Based (Systems) Architecting and Software Engineering Approach –MBASE/ RUP Activity / Process Model –WinWin Spiral model –Risk-driven strategy –Iterative refinement –Stakeholder commitment at anchor-point milestones –Emphasis of system and life-cycle issues. The MBASE Guidelines provide content, format and templates for project artifacts –Project Artifacts Operational Concept, System Requirements, System Architecture, Lifecycle Plan, Feasibility Studies Plans and reports for constructions and Transition phase Software tools –The Easy WinWin negotiation tool –Effort reporting tool –Risk identification tool –USC COCOMO II –COCOTS

2/13/07(c) USC-CSSE6 Problems in MBASE Problems with MBASE –MBASE is designed for large industry projects –Hugh Amount of effort in documentation –Less time to focus on project implementation Strategies –Analyze data from our project repositories –Shape up MBASE  leanMBASE

2/13/07(c) USC-CSSE7 Solutions in LeanMBASE –Minimum-essential documentation –Tailor up if proper –More time in project implementation Fall 2005, –the 260 pages of MBASE Inception and Elaboration phase Guidelines were replaced with 90 pages of LeanMBASE Guidelines.

2/13/07(c) USC-CSSE8 LeanMBASE Same approach, same documentation set, same tool but light-weight Identify high-value activities, balance the workload of a development Off-campus V&V  IIV&V (Integrated Independent) involve more to the team In Fall 2006, add one new document (SID: Supporting Information Document) as package header; combined all similar information.

2/13/07(c) USC-CSSE9 Outline Introduction Research Questions MBASE & its problem LeanMBASE –Content Comparison –Performance Comparison Effort Comparison Client Evaluation Comparison Conclusion Future Work

2/13/07(c) USC-CSSE10 Content Comparison I Operational Concept Definition (OCD); System and Software Requirements Definition (SSRD); System and Software Architecture Description (SSAD); Life Cycle Plan (LCP); Feasibility Rationale Document (FRD) Inception & Elaboration Document Size Comparison: LeanMBASE: smaller size; except SSAD in fall 2005 InceptionElaborationConstruction Transition

2/13/07(c) USC-CSSE11 Content Comparison II Plans and Reports in Construction and Transition Phase Peer Review Plan (PRP) Peer Review Report (PRP) Iteration Plan (IP) Iteration Assessment Report (IAR) Quality Management Report (QMP) Test Plan (TP) Test Description and Result (TDR) Release Description (RD) Transition Plan (TRP) Support Plan (SP) Packaged Tools and Procedures (PTP) Regression Test Package (RTP) User Manual (UM) Construction and Transition Document Size Comparison; Not much different in CTS documents; Room for improvement

2/13/07(c) USC-CSSE12 Effort Comparison Average number of hours spent for documentation: Less Effort, except SSAD in Fall 2005 Average number of hour/page in documentation: Less number of hours per page; except SSRD in Fall 2006

2/13/07(c) USC-CSSE13 Client Evaluation Semester Average Score of Customer evaluation FallSpringTotal (20 pts) (40 pts) Fall 03 – Sp 04 : MBASE Fall 04 – Sp 05 : MBASE Fall 05 – Sp 06 : LeanMBASE Fall 06 – Sp 07 : LeanMBASE17.9n/a Average Uniformly high; relatively close

2/13/07(c) USC-CSSE14 Conclusion With LeanMBASE –Smaller document size –Less time in documenting –Comparably satisfied with the project result from Clients –Off-campus students or IIV&V personnel play more roles At least for small real-client student-team projects, the LeanMBASE more efficiently supports students in –developing software projects, –learning software development –providing a win-win result to all key stakeholders.

2/13/07(c) USC-CSSE15 Future Work Additional Comparison –Risk Comparison –Defect List Comparison Process Enhancement –Electronic Process Guideline (LeanMBASE Plug-Ins on EPF OpenUp Framework) –WikiWinWin –Mentor Program