Readings in Computer Science: Software Engineering No Silver Bullet Essence and Accidents of Software Engineering Frederick P. Brooks, Jr. Prepared by Jinzhong Niu June 12, 2015June 12, 2015June 12, 2015

NSBReadings in Computer Science: Software Engineering2 Frederick P. Brooks, Jr.  Kenan Professor of CS at Univ. of North Carolina - Chapel Hill  Achievements  IBM OS/360  “The mythical Man-Month”  Honors and Awards  A.M. Turing Award, ACM (1999)  National Medal of Technology (1985)

NSBReadings in Computer Science: Software Engineering3 About this paper  Proc. IFIP Congress 1986, Information Processing 86  IEEE Computer, Vol. 20, No. 4, Apr  The Mythical Man-Month, 2nd Edition, 1995  Software Engineering, edited by Merlin Dorfman and Richard Thayer, Wiley-IEEE Press  1st Edition, 1996  2nd Edition, 2002

NSBReadings in Computer Science: Software Engineering4 What are werewolves and silver bullets?  Werewolf  one of the oldest monster legends  popular movie topic  Silver bullet  the only thing that can kill werewolves Even a man who is pure at heart, And says his prayers at night, Can become a wolf when the wolfbane blooms, And the moon is full and bright. -- From “The Wolf Man”

NSBReadings in Computer Science: Software Engineering5 “Essence” and “Accident”  Essence  noun, the permanent as contrasted with the accidental element of being  the mental crafting of the conceptual constructs  Accident  noun, a nonessential property or quality of an entity or circumstance; appurtenant (rather than misfortune or occurring by chance )  the implementation process of conceptual constructs

NSBReadings in Computer Science: Software Engineering6 Overview  Software development is werewolf, but there is no silver bullet because of its essential difficulties. I.What is the nature of software development? (Why does it have to be hard?) II.Did past breakthroughs solve the problem? III.Is there any potential solution nowadays? IV.Will the problem be attacked in the future?

NSBReadings in Computer Science: Software Engineering7 Problem solving strategy 知己知彼，百战不殆。 —— 《孙子兵法》 Know your enemy and know yourself; in a hundred battles, you will never be defeated. --- SUN TZU ON THE ART OF WAR

NSBReadings in Computer Science: Software Engineering8 Why does software engineering have to be hard?  Outside  Computer hardware progress is an exception.  Inside  There are essential difficulties which are hard to be attacked.

NSBReadings in Computer Science: Software Engineering9 Essential difficulties  Complexity  Conformity  Changeability  Invisibility

NSBReadings in Computer Science: Software Engineering10 Essential difficulties: Complexity  A system is usually defined as a collection of components, which interact with one another.  Software is much more complex than any other human construct.  The number of elements  The interaction between elements

NSBReadings in Computer Science: Software Engineering11 Essential difficulties: Complexity --- Cont.  A variety of problems are caused.  Technical  Decrease of reliability, usability, extensibility, safety  Managerial  Difficulty of communication between team members  Difficulty of keeping a clear integrated overview and all the loose ends  Difficulty of personnel turnover due to tremendous learning and understanding burden

NSBReadings in Computer Science: Software Engineering12 Why high complexity?  Software varies.  A colorful world needs colorful software systems, because “software has become the dominant technology in many if not most technical systems. It often provides the cohesiveness and data control that enable a complex system to solve problems.” [SwSE, Richard Thayer]  A single piece of software involves high complexity.  High conformity

NSBReadings in Computer Science: Software Engineering13 Essential difficulties: Conformity  Unlike physics where a terrible but invariable complexity exists, software has to conform many human institutions and system interfaces, the number of which is still swelling all the time.  Redesign of the software alone cannot simplify out the complexity.

NSBReadings in Computer Science: Software Engineering14 Essential difficulties: Changeability  Software is constantly subject to pressures for change.  Successful software DOES change frequently.

NSBReadings in Computer Science: Software Engineering15 Essential difficulties: Changeability --- Cont.  Why?  Necessity: Software embodies function, which most feels the pressures of change in a system.  Successful software is hoped to function over time.  It is hoped to function in new domains.  Feasibility: Software, pure thought-stuff, is infinitely malleable.

NSBReadings in Computer Science: Software Engineering16 Essential difficulties: Invisibility  Software is invisible in the sense that it is not inherently embedded in space.  Software structure is difficult to visualize in a hierarchical fashion.

NSBReadings in Computer Science: Software Engineering17 Did past breakthroughs solve the problem?  No. What they attacked are accidental difficulties not essence.  Give me some examples!  High-level languages  Time-sharing  Unified programming environments

NSBReadings in Computer Science: Software Engineering18 High-level languages The development of high-level languages is credited with  at least a factor of five in productivity,  concomitant gains in reliability, simplicity, and comprehensibility.  It, however, eliminates only the complexity related to lower level constructs that are not inherent in software.  The level of our thinking about data structures, data types, and operations is steadily rising, but at an ever decreasing rate, and approaches closer and closer to the sophistication of users.

NSBReadings in Computer Science: Software Engineering19 High-level languages --- Cont.

NSBReadings in Computer Science: Software Engineering20 Time-sharing Time-sharing eliminates the slow turnaround of batch programming, and keeps fresh in mind the grasp of a complex system.  The benefit of time-sharing is to be boundary due to the human threshold of noticeability.

NSBReadings in Computer Science: Software Engineering21 Unified programming environments Unified programming environments enable related individual tools to work together in an automatic manner. They thus free programmers from the burden of various manual operations.  By its very nature, the fruit is and will be marginal.

NSBReadings in Computer Science: Software Engineering22 Is there any potential solution nowadays?  Ada and other high-level language advances ?  Object-oriented programming ?  Artificial Intelligence ?  Expert Systems ?  “Automatic” programming ?  Graphical programming ?  Program verification ?  Environments and tools ?  Workstations ?

NSBReadings in Computer Science: Software Engineering23 Ada Ada, one of the most touted recent development, not only reflects evolutionary improvements in language concepts, but indeed embodies features to encourage modern design and modularization.  Nevertheless, it is just another high-level language and will not prove to be the silver bullet.

NSBReadings in Computer Science: Software Engineering24 Object-oriented programming  Two orthogonal concepts representing real advances:  abstract data types  hierarchical types  OO Concepts:  encapsulation  abstraction  inheritance  polymorphism  dynamic binding

NSBReadings in Computer Science: Software Engineering25 Object-oriented programming --- Cont. OO represents real advances in the art of building software.  Nevertheless, they remove only accidental difficulties from the expression of the design, rather than the design itself.

NSBReadings in Computer Science: Software Engineering26 Artificial Intelligence  Terminological chaos – Two definitions:  AI-1: The use of computers to solve problems that previously could only be solved by applying human intelligence.  AI-2: The use of a specific set of programming techniques known as heuristic or rule-based programming. (expert system)

NSBReadings in Computer Science: Software Engineering27 Artificial Intelligence --- Cont. AI advancements facilitate HCI (Human Computer Interface).  However, the hard thing about building software is deciding what to say, not how to express.

NSBReadings in Computer Science: Software Engineering28 Expert Systems  Definition:  a program containing a generalized inference engine and a rule base, takes input data and assumptions, explores the inferences derivable from the rule base, yields conclusions and advice, and explains its results by retracting its reasoning for the user Advantages:  Inference-engine technology is application-independent.  The application-peculiar materials are encoded in the rule base in a uniform fashion, which regularizes the complexity of the application itself.

NSBReadings in Computer Science: Software Engineering29 Expert Systems --- Cont. User User Interface Inference Engine Input data Knowledge Base (Rules, Facts) advices

NSBReadings in Computer Science: Software Engineering30 Expert Systems --- Cont. Possible benefits:  Expert systems in software engineering field  Building software in the way expert systems work  Difficulties  How to generate automatically the diagnostic rules from program-structure specification  How to extract expertise and distill it into rule bases

NSBReadings in Computer Science: Software Engineering31 “Automatic” programming  Automatic programming is actually a euphemism for programming with a higher-level language so that a solution could be given more easily. There are some exceptions which have favorable properties:  Relatively few parameters are involved.  Many solutions are available.  Explicit rules are known to select solutions.  It is hard to generalize such special cases for the ordinary software systems.

NSBReadings in Computer Science: Software Engineering32 Graphical programming Computer graphics, which has been applied successfully in other fields, seems to be able to play a role in software design.  Nothing convincing has ever emerged from this approach.  The flowchart, considered as the ideal program-design medium, is a very poor abstraction of software structure.  The screens of today are too small to show detailed software diagrams.  In its nature, software is very difficult to visualize.

NSBReadings in Computer Science: Software Engineering33 Program verification Program verification seems promising to avoid immense effort upon implementation and testing by eliminating errors in the design phase.  No magic!  Verifications are so much work that only a few programs have been verified.  Verification cannot eliminate errors totally since mathematical proofs can also be faulty.  Specification, the baseline of verification, is usually incomplete and inconsistent.

NSBReadings in Computer Science: Software Engineering34 Environments and tools  We have discussed this issue.

NSBReadings in Computer Science: Software Engineering35 Workstations More powerful computers surely facilitate software development.  But nowadays time of thinking, instead of waiting for computers' response, is the dominant activity of programmers. Magical enhancement thus cannot be expected.

NSBReadings in Computer Science: Software Engineering36 Will the problem be attacked in the future?  The conceptual components of the task are now taking most of time.  We must consider those attacks that address the essence of the software problem.

NSBReadings in Computer Science: Software Engineering37 Will the problem be attacked in the future? --- Cont.  Well, there may be some copper bullets:  Buy versus build  Requirements refinement and rapid prototyping  Incremental development  Greater designers

NSBReadings in Computer Science: Software Engineering38 Buy versus build  It is common practice to buy off-the-shelf products nowadays due to the following reasons:  PC revolution has created many mass markets for software, which, together with zero replication cost of software, stirred the motivation for software companies to produce more and better software products.  Applicability of software is enhanced with the generalization of software tools and the constantly decreasing hardware/software cost ratio.

NSBReadings in Computer Science: Software Engineering39 Requirements refinement and rapid prototyping  It is hardest to decide detailed technical requirements. Unfortunately even the clients themselves do not exactly know what they want.  So iterative extraction and refinement of product requirements are necessary.

NSBReadings in Computer Science: Software Engineering40 Requirements refinement and rapid prototyping --- Cont.  A client cannot specify completely, precisely, and correctly the exact requirements of a modern software product before trying some versions of the product.  Rapid prototyping may give clients a first-hand feel of what the product will be and a check for consistency and usability.

NSBReadings in Computer Science: Software Engineering41 Incremental development  To develop software that has a comparable complexity as human brain, a similar process should be followed, i.e. incremental development.  Advantages:  The approach necessitates top-down design, thus allowing easy backtracking and detecting fundamental defects as early as possible.  An always working system stirs enthusiasm.

NSBReadings in Computer Science: Software Engineering42 Greater designers  People is the key factor of solving problems.  Steps have been taken to raise the level of our practice from poor to good.  Curricula  Literature  Research organizations

NSBReadings in Computer Science: Software Engineering43 Greater designers --- Cont.  The proposed next step is to develop ways to grow great designers.  Why?  Creative minds present state-of-the-art works, the benefits of which are order-of-magnitude compared with the average practices.  How?  Identify top designers as early as possible  Assign a career mentor to be responsible for the development of the prospect  Work out a career-development plan for each prospect  Provide opportunities for designers to interact with and stimulate each other

NSBReadings in Computer Science: Software Engineering44

NSBReadings in Computer Science: Software Engineering45 Bullets towards NSB 1.We cannot abstract away the complexity without abstracting away the essence?  “Divide and conquer” strategy  Is a hierarchical model of software possible? We cannot always visualize software in hierarchical graphs? 2.The benefit of time-sharing is boundary? 3.The hardest single part of building a software system is deciding precisely what to build?  How about design?