1. Software Engineering Process 2007

Motivation for Software Engineering IT projects have a terrible track record A 1995 Standish Group study (CHAOS) found that only 16.2% of IT projects were successful and over 31% were canceled before completion, costing over $81 B in the U.S. alone The need for IT projects keeps increasing In 2000, there were 300,000 new IT projects In 2001, over 500,000 new IT projects were started

The Four “P’s” of Software Engineering People (by whom it is done) Process (the manner in which it is done) Project (the doing of it) Product (the application artifacts)

People “It’s always a people problem” Gerald Weinberg, “The Secrets of Consulting” Developer productivity: 10-to-1 range Improvements: Team selection Team organization Motivation Teams: 5-to-1 range

People 2 Other success factors Matching people to tasks Career development Balance: individual and team Clear communication

Optimal Size for Interaction (Approximate) Effectiveness per developer Developer communicates regularly with no one. No communication time lost, but developer is too isolated and has no help. 3 Number of people with whom developer must frequently interact Key: = engineer

Optimal Size for Interaction (Approximate) Effectiveness per developer Approximate optimal range Developer communicates regularly with eleven people. Communication time outweighs benefits of interaction Developer communicates regularly with no one. No communication time lost, but developer is too isolated and has no help. 3 7 Number of people with whom developer must frequently interact Key: = engineer

Organizations for Larger Projects Team of leaders

Quality Application must satisfy predetermined standards. Methods to attain quality goals: Inspection team-oriented process for ensuring quality applied to all stages of the process. Quality

Quality Application must satisfy predetermined quality level. Methods to attain quality level: Inspection team-oriented process for ensuring quality applied to all stages of the process. Formal methods mathematical techniques to convince ourselves and peers that our programs do what they are meant to do applied selectively Testing at the unit (component) level at the whole application level Project control techniques predict costs and schedule control artifacts (versions, scope etc.) Quality

Set of activities carried out to produce an application Process Set of activities carried out to produce an application

A Defined Process

Process 2 Types: Management & Technical Development fundamentals Quality assurance Risk management Lifecycle planning cut time-to-market Improve quality

Process 2 Customer orientation Process maturity improvement Rework avoidance

Process Development sequence: Process frameworks: Standards: Waterfall Iterative Process frameworks: Personal Software ProcessSM Team Software ProcessSM Capability Maturity ModelSM -- for organizations Standards: Institute of Electrical and Electronic Engineers International Standards Organization ...

What Is a Project? A project is “a temporary endeavor undertaken to accomplish a unique product or service” (PMBOK® Guide 2000, p. 4) Attributes of projects unique purpose temporary require resources, often from various areas should have a primary sponsor and/or customer involve uncertainty

The Triple Constraint Every project is constrained in different ways by its Scope goals: What is the project trying to accomplish? Time goals: How long should it take to complete? Cost goals: What should it cost? It is the project manager’s duty to balance these three often competing goals

The Triple Constraint

The Triple Constraint Fast, cheap, good. Choose two.

The Triple Constraint Know which of these are fixed & variable for every project

The Variables of Project Management Can somewhat vary the following factors. 1. The total cost of the project, e.g., increase expenditures 2. The capabilities of the product, e.g., subtract from a list of features 3. The quality of the product, e.g., increase the mean time between failure 4. The date on which the job is completed. e.g., reduce the schedule by 20% e.g., postpone project's completion date one month The Variables of Project Management

Project Variables cost capability duration defect density Target: 100% $70K capability duration Target : 30 wks Target : 4 defects/Kloc defect density

Project Variables cost this project capability duration defect density Target: 100% cost Target : $70K Actual: 100% Actual: $90K this project capability duration Target : 30 wks Target : 4 defects/Kloc Actual: 20 wks Actual: 1 defect/Kloc defect density

Product The “tangible” dimension Product size management Product characteristics and requirements Feature creep management

Software Engineering Roadmap

Software Engineering Roadmap Plan configuration management - how to manage documents & code - document: SCMP Plan quality assurance - how to ensure quality - document: SQAP Identify corpor- ate practices - assess capability - specify standards - e.g., CMM level Plan verification & validation - verify the product satisfies requirements - validate each phase by showing it succeeded document: SVVP next chapter: Plan development process Plan project Analyze requirements Maintain Development phases Integrate & test system Design Implement Test units

Software Engineering Roadmap Plan configuration management - how to manage documents & code - document: SCMP Plan quality assurance - how to ensure quality - document: SQAP Identify corpor- ate practices - assess capability - specify standards - e.g., CMM level Plan verification & validation - verify the product satisfies requirements - validate each phase by showing it succeeded document: SVVP Plan development process Plan project Analyze requirements Maintain Development phases Integrate & test system Design Implement Test units

Typical Project Roadmap 1. Understand nature & scope of proposed product 2. Select the development process, and create a plan 3. Gather requirements 4. Design and build the product 5. Test the product 6. Deliver and maintain the product

Planning Determine requirements Determine resources Select lifecycle model Determine product features strategy

Tracking Cost, effort, schedule Planned vs. Actual How to handle when things go off plan?

Measurements To date and projected Alternatives Cost Schedule Effort Product features Alternatives Earned value analysis Defect rates Productivity (ex: SLOC) Complexity (ex: function points)

Project Phases

Lifecycle Relationships

History (very shortly)

Structured Programming Function definition handleAccount(…) getDetailsFromUser(…) getAccount(…) doTransaction(…) …… Function definition getDetailsFromUser (…) getName(…) …... Function definition getAccount(…) getFirstName(…) TOP DOWN

Object Orientation Account Transaction Customer Real world concepts Skljkvjkvjfkavjafkk saovjsdvjfvkfjvkfjk Direct correspondence Software design entities Account getDetails() Transaction execute() Customer getFirstName()

Components Software may be built using existing services Reusable Software Make-or-(better) Buy Design for Changeability, Exchangeability

What is a Component? Definition: A component denotes a self-contained entity that provides its functionality via standard interfaces uses functionality from its environment via standard interfaces and may support builder tools in plugging components and applications together

Komponent: peidetud realisatsioon

Hajuskomponent

Expectations for Software Engineering Process

Five Key Expectations 1. Predetermine quantitative quality goals Used for process development Part of the project 2. Accumulate data for subsequent use 3. Keep all work visible 4. A. Design only against requirements B. Program only against designs C. Test only against requirements and designs Aspect of the product Influenced by people 5. Measure quality

Artifacts and Roles SW Dev Proc term Symbol & examples Artifacts: the entities that software engineering deals with. Document Model -- a view of the application (design etc.) Component -- physical (source code etc.) Workers: responsibilities allocated to people (roles). Worker Worker instance (e.g., Joe Smith)

Software Engineering Process alternatives

Main Phases of Software Process Requirements Analysis (answers “WHAT?”) Specifying what the application must do Design (answers “HOW?”) Specifying what the parts will be, and how they will fit together Implementation (A.K.A. “CODING”) Writing the code Testing (type of VERIFICATION) Executing the application with test data for input Maintenance (REPAIR or ENHANCEMENT) Repairing defects and adding capability

Software Process Phases Requirements Analysis: Text produced e.g., “ … The application shall display the balance in the user’s bank account. …” Design: Diagrams and text e.g., “ … The design will consist of the classes CheckingAccount, SavingsAccount, …” Implementation: Source and object code e.g., … class CheckingAccount{ double balance; … } … Testing: Test cases and test results e.g., “… With test case: deposit $44.92 / deposit $32.00 / withdraw $101.45 / … the balance was $2938.22, which is correct. …” Maintenance: Modified design, code, and text e.g., Defect repair: “Application crashes when balance is $0 and attempt is made to withdraw funds. …” e.g., Enhancement: “Allow operation with Pesos.” Software Process Phases

The Waterfall Model Concept analysis Analysis Design Implementation & unit testing Integration System testing Maintenance

The Waterfall Software Process time Milestone(s) Release product X Requirements Analysis Two phases may occur at the same time for a short period Design Implementation Phases (activities) Testing Maintenance

Why a Pure Waterfall Process is Usually Not Practical Don’t know up front everything wanted and needed Usually hard to visualize every detail in advance We can only estimate the costs of implementing requirements To gain confidence in an estimate, we need to design and actually implement parts, especially the riskiest ones We will probably need to modify requirements as a result We often need to execute intermediate builds Stakeholders need to gain confidence Designers and developers need confirmation they're building what’s needed and wanted Team members can't be idle while the requirements are being completed Typically put people to work on several phases at once

Spiral Development Sequence Product: requirements specifications Product:class models + Step n: Analyze requirements Step n+1: Design complete targeted requirements Step n+2: Implement Step n+3: Test Product: code + Product: test results +

The Spiral Process time M I L E S T O N E S Intermediate version* completed X Product released X Iteration # 1 2 3 Requirements analysis 1 2 3 Design 1 2 3 Coding 1 2 3 Testing 1 2 3 *typically a prototype

Iterative Development Iteration No. 1 2 3 867 868 Update SPMP1 Test whole Integrate Update Test documentation Test units Update source code Implement Design Update SDD2 Analyze requirements Update SRS3 1 Software Project Mangement Plan 2 Software Design Document 3 Software Requirements Specification

Iterative development

The Unified (Software Development) Process: Classification of Iterations Inception iterations: preliminary interaction with stakeholders primarily customer users financial backers etc. Elaboration iterations : finalization of what’s wanted and needed; set architecture baseline Construction iterations : results in initial operational capability Transition iterations : completes product release

Classification of iterations (Classically called “phases”) UP Terminology (1) Classification of iterations Individual iteration Inception Elaboration Construction Transition Prelim. iterations Iter. #1 Iter. #n Iter. #n+1 Iter. #m Iter. #m+1 Iter. #k .. ….. … Requirements Analysis UP calls these “disciplines” (Classically called “phases”) Design Implemen- tation Test

Requirements analysis UP Terminology (2) Requirements Analysis Design Implementation Test Requirements analysis UP Terminology Classical Terminology Integration

Unified Process Matrix Inception Elaboration Construction Transition Prelim. iterations Iter. #1 Iter. #n Iter. #n+1 Iter. #m Iter. #m+1 Iter. #k .. ….. ….. Requirements Analysis Amount of effort expended on the requirements phase during the first Construction iteration Design Implemen- tation Test

The Six UP Models (Views of the Application) Use-case model Test model Analysis model Implementation model Design model Deployment model

Project Documentation SVVP software validation & verification plan Verification & validation SQAP software quality assurance plan Quality assurance SCMP software configuration management plan Configuration SPMP software project management plan Project status Customer-oriented SRS software requirements specifications Requirements Developer-oriented Architecture SDD software design document Design Detailed design Code Source Code Project Documentation STD software test documention Testing Operation User’s manual

Software Engineering Process Quality

are we building the thing right? Validation: Meaning of “V&V”(Boehm) Verification: are we building the thing right? Validation: are we building the right thing?

QA QA Involvement 2. QA reviews 1. QA Develops process for 1. Specify how to manage project documents 2. Identify process QA Involvement 2. QA reviews process for conformance to organizational policy 1. QA Develops and/or reviews configuration management plans, standards ... QA 3. QA develops and/or reviews provision for QA activities 5. QA reviews, inspects & tests 4. QA reviews, inspects & tests 5. Deliver & main- tain the product 4. Design and build 3. Plan

Inspection Process & Example Times Nominal process 1. PLANNING OVERVIEW Non-nominal process 2. PREPARATION CAUSAL ANALYSIS 3. INSPECTION 4. REWORK Inspection Process & Example Times 5. FOLLOW-UP 6. IMPROVE PROCESS

Time/Costs per 100 LoC* -- one company’s estimates Planning 1 hr  (1 person) [ Overview 1 hr  (3-5) ] Preparation 1 hr  (2-4 people) Inspection meeting 1 hr  (3-5 people) Rework 1 hr  (1 person) [ Analysis 1 hr  (3-5) ] Total: approx. 7 - 21 person-hours * lines of non-commented code

Hours Per Defect: One estimate If defect found ... … at inspection … at integration time time Hours to .. .. detect 0.7 to 2 0.2 to 10 .. repair 0.3 to 1.2 9+ Total 1.0 to 3.2 9.2 to 19+

Prepare For & Conduct Inspections One way to ... Prepare For & Conduct Inspections 1. Build inspections into the project schedule plan to inspect all phases, starting with requirements allow for preparation (time consuming!) & meeting time 2. Prepare for collection of inspection data include # defects per work unit (e.g., KLOC), time spent develop forms: include description, severity and type decide who, where, how to store and use the metric data default: appoint a single person to be responsible failure to decide usually results in discarding the data 3. Assign roles to participants Three adequate (author; moderator/recorder; reader) Two far better than none (author; inspector) 4. Ensure every participant prepares bring defects pre-entered on forms to inspection meeting Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Software Engineering Process Documentation

Project Documentation SVVP software validation & verification plan Verification & validation SQAP software quality assurance plan Quality assurance SCMP software configuration management plan Configuration SPMP software project management plan Project status

Example of Documentation Set (with Dynamic Content shown) STP software test plan Test results* SPMP software project management plan Project status* SRS software requirements specifications Direction of hyperlink Updates* References to all other documents Source Code SCMP software configuration management plan SDD software design document Configuration* Updates* *Dynamic component

Configuration Items (CI’s) Units tracked officially down to the smallest unit worth tracking includes most official documents Payroll v. 0.3.4.2 Payroll v. 0.3.4.3 Payroll v. 0.4.1 S6 A1 C4 D5 E3

Configuration Items (CI’s) Units tracked officially down to the smallest unit worth tracking includes most official documents Payroll v. 0.4.1 Payroll v. 0.3.4.2 Payroll v. 0.3.4.3 S6 S7 S7 A1 A1 A1 C4 C4 C4 D5 D5 D5 E3 E3 E3 F1

Summary Software engineering an extensive challenge Major process models: waterfall; spiral; incremental Capability frameworks: CMM; TSP; PSP Quality is the professional difference metrics to define inspection throughout rigorous testing include continuous self-improvement process Documentation: SCMP, SVVP, SQAP, SPMP, SRS, SDD, STP, Code, User’s manual