1. Trustworthy Systems Through Quantitative Software Engineering Larry Bernstein Stevens Institute of Technology Castle Point, Hoboken, NJ 07030 USA

2. Trustworthy Software is: Safe: Does no harm Reliable: No crash or hang. Secure: No Hacking Possible

3. What is a Requirement? A property that must be exhibited by a system to solve some problem. Requirements may be –Functional providing product capabilities –Non-Functional constraining the implementation

4. System Performance Resulting from Robust Requirements vs. Discrete Specifications Volume Dynamic Range Ideal Discrete Specifications Robust Requirements

5. Relative Project Costs Relative Cost Range x 1.25x 1.5x 2x 4x 0.8x 0.5x 0.8x 0.67x Phases and Milestones Feasibility Concept of Operation Plans and Requirements Requirements Specifications Product Design Product Design Specifications Detailed Design Detailed Design Specifications Development and Test Accepted Software

6. Top Ten Software Risk Items CategoryRisk Item People1. Personnel Shortfalls 2. Unrealistic Schedules and Budgets Requirements3. Developing the Wrong Software Functions 4. Developing the Wrong User Interface 5. Gold Plating 6. Continuing Stream of Requirements Changes Externalities7. Shortfalls in Externally-Furnished Component 8. Shortfalls in Externally-Performed Tasks Technology9. Real-Time Performance Shortfalls 10. Straining Computer Science Capabilities

7. Highlights of Quantitative Approach Lambda Protocol Overlaps with Systems Engineering Industrial Strength Requirements for Software Intensive Systems-of-Systems

8. QSE Lambda Protocol Prospectus Measurable Operational Value Prototyping or Modeling sQFD Schedule, Staffing, Quality Estimates ICED-T Trade-off Analysis

9. Universal Software Engineering Equation Reliability (t) = ℮ -k t when the error rate is constant and where k is a normalizing constant for a software shop and = Complexity/ effectiveness x staffing

10. Conditions That Cause Unreliability Poor Algorithms Missing Deadlines Roundoff Error Build Up Memory Leaks Broken Pointers Register Misuse

11. QSE Characteristics Solving the right problem the right way Certified against requirements. Certified against problem Bounded execution domain

12. Case Study: SchedulerPro Prospectus User friendly, efficient interface for students to create and modify class schedules. Features: –Visual schedule creation and editing –Schedule suggestion –Schedule comparison view –Monitor closed-out sections

13. SchedulerPro Prototype Screen

15. SchedulerPro Notification Emails

16. Measurable Operational Value SchedulerPro MOV Course scheduling and registration time reduced by an average of 20 minutes per student per semester.

17. SchedulerPro Functional Goals Schedule Classes and Personal Time  Searching  Course Placement  Course Detail Viewing  Course Removal  Scheduling Personal Blocks  Notification (optional)  Course Suggestions (optional)

18. Functional Requirements Search available classes by: Same professor Similar time Same or equivalent class but different sections Register and track registrations Color classes and arbitrary time-blocks by user choice

19. SchedulerPro Nonfunctional Requirements Integrate with “Web for Students’ and existing authentication systems and avoid incompatibilities Allow schedules to be saved/accessed from a server or local file Provide a scaled time-accurate visual representation of the schedule

20. More Non-functional requirements Make schedules available even if the application is down, provided an internet connection is available Perform some functions without a live connection to the ‘Web for Students’ registrar web site Make compatible with all popular browsers Display section states and print schedules without loss of detail

21. sQFD Features / Functions Class Filters Allocate non- class time Long term information availability Authenticate Makes scheduling classes easier 8362 19 Makes scheduling a semester easier 7982 26 Find schedules in one place 1157 14 Total16131911 59

22. SchedulerPro Product Reliability Two hours of unavailability allows for daily backups, service, and reboots of the system Connections to server are minimized, reducing overall activity on the server

23. Parnas reliability checklist Failures in communication, secondary storage, memory, or any hardware that may interrupt a transaction: The SQL Server DBMS will not commit incomplete transactions. User will be notified of the error, and will have to redo the transaction. Operator error: Important operations are confirmed before they are completed to avoid large accidental errors.

24. SchedulerPro Estimate of Reliability R(t) = 1 - F(t) F(t) = P(T ≤ t) During load testing, we discovered the test server can support 1500 user queries a minute. P(failures/query) = 55/1500 = 0.036P(failures/query) = 55/1500 = 0.036 Thus, F(t) = 3.6%, which means the software is 96.4% reliable

25. SchedulerPro Reliability Estimate 1/ λ = MTTF = εE/kC k = scaling constant = 1 C is complexity = 2.78 E is the development effort = 36.4 ε is the expansion factor = 1.5 λ = 0.05 t is the continuous execution time for the software R(t) = 95.12%

26. Complexity Chart - Client Project Type: online transaction Problem Domain: 2 Architecture Complexity: 3 Logic Design – Data: 2 Logic Design – Code: 3 Total Score: 10 Complexity = (10/18) * 5 = 2.78

27. Complexity Chart - Server Project Type: online transaction Problem Domain: 1 Architecture Complexity: 2 Logic Design – Data: 2 Logic Design – Code: 2 Total Score: 7 Complexity = (7/18) * 5 = 1.94

28. Complexity Chart - Overall Project Type: client/server Problem Domain: 2 Architecture Complexity: 3 Logic Design – Data: 2 Logic Design – Code: 3 Total Score: 10 Complexity = (10/18) * 5 = 2.78

29. Jan. Function Point Est. Jan. Function Point Est. FunctionLow (L)Average (A)High (H)Total Outputs130 19 Inquiries841 49 Inputs571 41 Internal Files320 24 External Interfaces210 10 Total UFP143 Adjustment Factor0.99 Total AFP141

30. April Function Points Est. FunctionLowAverageHighTotal Outputs1019 Inquiries3009 Inputs23018 Internal Files31031 External Interfaces 11012 Total UFP79 AFP82

31. History of Function Points DateAFPProject Length* Projected Finish* January 2714119.7 staff months August 2006 February 2410414.4 staff months March 2006 April 17828.5 staff months May 2006 *Using COCOMO Model

32. Web Points FunctionTotal Web Objects Outputs11 Inquiries9 Inputs18 Internal Files31 External Interfaces12 # Multimedia files6 # Web building blocks7 # Scripts3 # Links0 Total Web Objects97 Effort = A * cd i (Size) P1 Because it is a small web project, assume: A=2.1, B=2.0, P1=1.00, and P2=.5 Language Expansion Factor = 25 From this, we estimate 2.43 KLOC Effort = A* KLOC = 2.1 * 2.43 = 5.103 staff-months Duration =B * Effort P2 = 2.0 * 5.103.5 4.518 calendar months = 4.518 calendar months

33. ICED-T Scheduling by: IntuitiveConsistentEfficientDurableThoughtful Paper32223 Excel32333 School Scheduler 34434 SchedulerPro44545

34. Creeping Featurism Endemic to the Software Industry –Occurs on more than 70% of all applications of over 1000 function points From a 60 project sample –Average creep was 35% –Maximum observed was 200% –Creeping requirements change about 1% per month For a 3 year project, 1/3 of the delivered requirements would have been added after requirements were initially defined Rate of Requirements change is higher than for other forms of engineering (electrical, mechanical, civil)

35. Root Causes of Creeping Requirements Uncertainty in resolving true user needs For multi-year projects, changes in normal business environment Failure to adopt methodologies that limit the risk associated with creeping requirements Primitive fundamental technologies for exploring and modeling requirements Failure to use technology to measure the impact of creeping requirements Engineering trade-off analysis is impossible

36. Requirements Management Establish and maintain a business case to support funding Strategic linkages to business and technology organizations –AVOID SHELFWARE Continuous customer agreement on requirements Requirements agreement used as a basis for estimating, planning, implementing and tracking FORMAL COMMITMENT PROCESS

37. SchedulerPro Installation Plan Installation 1. Third Party Software Required Scheduler Pro requires the following products to be already installed on the target machine. Please consult the documentation of each product for installation instructions specific to each. - Windows 2000, XP, or 2003 Server - Microsoft IIS, version 5.0 or higher - Microsoft.NET, version 1.1 - Microsoft SQL Server 2000 - Message Queuing Service (Windows component) - ASP.NET State Service

38. 2. Installing the Scheduler Pro application To install Scheduler Pro, please follow these steps: I. Setting up the web site 1. Create a virtual directory for the site in IIS 2. Copy the contents of the “site” folder to the directory on your hard drive represented by the IIS virtual directory II. Setting up the database 1. Using the SQL Server Enterprise Manager tool, attach the database located under the “sqldb” directory. 2. Create a new user account for accessing this database, and give it read/write access for the database. III. Installing the Suggest, Notified, and Data Loader Windows Services Open the directory titled “services”. Run each of these files: - InstallSuggestService.exe - InstallNotifierService.exe - InstallDataLoaderService.exe Each installer will properly install the service. The Data Loader installer will also ask you for the location and name of the course data file it will load. SchedulerPro Installation Plan

39. 3. Configuring Your Installation Open the directory where you installed the site files (step 2A above). Open the file “web.config”, and find the properties labeled “DBLocation”, “DBUser”, “DBPassword”, and “SecretAuthenticateKey”. These are set to default values that you should modify. Make sure that the location, username, and password match those you set up in SQL Server in step 2B above. Set SecretAuthenticateKey to the random string you are using in your authentication web page. Technical Information Scheduler Pro uses HTTP and XMLTHTTP protocols: - HTTP: As a typical client/server application, an HTTP request is sent to server. Server processes the request and returns HTTP response. - XMLHTTP: client opens XMLHTTP connection to the server. Server executes query on the database and returns needed data. Refer to Development view and process view from the “4+1” architecture views for further technical details.

40. People Software Trustworthiness depends on people: I propose that customers insist that software products identify a Software Architect and Software Project Manager in their contracts

41. Software Architect: Affirms that the software product solves the customer’s problem Affirms that the software product is suitably reliable, easy-to-use, extendible, not harmful and robust. That it is trustworthy. Affirms that the requirements are valid.

42. Software Project Manager: Affirms that the software was successfully tested against the requirements. Affirms and identifies the good software engineering processes were used in the software development and integration. Affirms that the project is within budget, on- time and performs satisfactorily.