“Software's Chronic Crisis” by W. Wayt Gibbs Presented by Daniel Thomasset EECS 810 - Fall 2005

Organization Software's chronic crisis Factors contributing to the crisis challenges of large software case studies impacts of software failures The need for software engineering software engineering defined Software engineering techniques CMM Standardization Concluding remarks

Software's Chronic Crisis The software crisis is chronic chronic - of long duration, continuing - marked by frequent reoccurance crisis - a crucial state of affairs in which a decisive change is impending; especially one with the possibility of an undesirable outcome Definitions: Merriam-Webster Dictionary, online

Challenges of Large Software Distributed and realtime systems distributed software is complex and has more paths of failure Human comprehension No single person can completely comprehend a large project Growth in hardware complexity hardware complexity doubles every 18 months Lack of standardization interfaces and methods for software construction can differ Lack of measurement time and cost are hard to estimate without empirical knowledge of the organization's capabilities

Case Study: Denver Airport A complex distributed software system baggage delivered by 4000 automated “telecars” 100 computers controlled movement using electric eyes, bar-code sensors, and radio receivers cost $193M for initial implementation Over budget, operating failure failure delayed airport opening one year (cost >$1M per day) canceled in August 2005 Reasons for failure high maintenance costs: $1M per month couldn't handle real-world errors Sources: Johnson, 2005 and Gibbs, 1994

Impacts of Software Failures Business impacts from IBM Consulting Group survey 55% of projects cost more than expected 68% took longer than estimated to complete 88% of projects had to be substantially redesigned Failed projects are not used 75% of large projects are operating failures or not used at all Public safety can be affected software controls trains, aviation, life support systems, and nuclear power plants

Case Study: Federal Aviation Administration (FAA) Advanced Automation System (AAS) replacement air-traffic control system contracted to IBM in 1983 with a budget of $2.6 billion >1M lines of code, 100s of computers Over budget, late, and unfinished in 1996, General Accounting Office (GAO) reports 57% of the budget was wasted 2/3 of project is canceled, the rest is late Reasons for failure FAA assumed that IBM would use engineering techniques GAO reports that “human factors” were the main reason for failure Sources: House, 2001 and Gibbs, 1994

Need for Software Engineering Software development is preindustrial “It's like musket making was before Eli Whitney” artisans using little standardization each piece is unique Consistent software quality requires a process “If we are ever going to lick this software crisis, we're going to have to stop this hand-to-mouth every-programmer-builds-everything-from-the-ground-up, preindustrial approach” Quotes: Brad Cox in Gibbs, 1994

Software Engineering First defined in1968 by the NATO Science Committee 50 software experts met to solve the software crisis defined software engineering as “the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software”

The Engineering Process Important aspects of an engineering discipline systematic - establishes standard practices - allows repeatable results quantifiable - allows comparison of systems - allows estimation of time, cost, The process of improvement a systematic and quantifiable system allows for continuous process improvement the cycle 1. establish a system, do a project, measure the results 2. evaluate and refine the system 3. goto step 1

Capability Maturity Model The Capability Maturity Model (CMM) developed by the Software Engineering Institute (SEI) a quantifiable engineering process applied to software development that “provides a vision of software engineering and management excellence” defines levels of maturity from 1 to 5. updated version called Capability Maturity Model Integration (SEI, 2005) Results of Raytheon's transition from Level 1 to 3 most projects completed ahead of schedule and under budget productivity doubled savings of $7.80 for every dollar invested in CMM Quote: David Zubrow in Gibbs, 1994

CMM Levels

Other Techniques Formal Methods Growing Software Cleanroom mathematically prove that the algorithm is correct Growing Software start simple and grow until complete Cleanroom combines formal methods, statistical quality control, and evolutionary approach analogous to a hardware clean room - don't let in the bugs

Standardization Standard software pieces Standard processes Results interfaces become simpler allows standard tests Standard processes forces all software developers to use best-practices may require government involvement and licensing Results increases reliability reduces repeated mistakes facilitates budget estimation

Summary Software's chronic crisis Factors contributing to the crisis challenges of large software case studies impacts of software failures The need for software engineering software engineering defined Software engineering techniques CMM standardization

Conclusions Software development is hard complexity will always cause challenges Software engineering improves the process continuous improvement possible with method and measurement Industry is making progress in 1994, two CMM level 5 organizations today, eighty-five CMM level 5 organizations (SEI, 2005) Slow, directed growth there's no “silver bullet” research is required to make improvements in the process

Bibliography Gibbs, W. Wayt, “Software's Chronic Crisis”, Scientific American, September 1994. House of Representatives, Aviation Subcommittee, “FAA's efforts to modernize the Air Traffic Control system”, March 14, 2001. Johnson, Kirk, “Denver Airport Saw the Future: It Didn't Work”, New York Times, August 27, 2005. http://www.nytimes.com/2005/08/27/national/27denver.html Software Engineering Institute, “Capability Maturity Model Integration (CMMI) Overview”, 2005 http://www.sei.cmu.edu/cmmi/adoption/pdf/cmmi-overview05.pdf