University of Southern California Center for Software Engineering C S E USC Barry Boehm, USC CS 510 Lecture Fall 2001 ( Agile Methods

University of Southern California Center for Software Engineering C S E USC ©USC-CSE2 Outline Silver bullets and lead bullets Information technology trends –The dwindling lead-bullet niche –Underlying world-view trends Agile methods –Example: eXtreme Programming (XP) –Counterpoint: A skeptical view Relations to CMM and CMMI –How much planning is enough?

University of Southern California Center for Software Engineering C S E USC ©USC-CSE3 Even if we can’t define what the “software problem” werewolf is, And we’ve seen that lead, bronze, iron, and steel bullets can’t kill it, We’re sure there’s a silver bullet that can ****** Fallacy: We can fix things we understand –“Accidental” software problems But we can’t fix things we don’t understand –“Essential” software problems The Silver Bullet Fantasy (Brooks, 1986)

University of Southern California Center for Software Engineering C S E USC ©USC-CSE4 Example: Conway’s Law and its Converse Conway’s Law (Datamation, 1968), extended The structure of a computer program...

University of Southern California Center for Software Engineering C S E USC ©USC-CSE5 Example: Conway’s Law and its Converse Conway’s Law (Datamation, 1968), extended The structure of a computer program... reflects the structure of the organizations that build and use it

University of Southern California Center for Software Engineering C S E USC ©USC-CSE6 Converse of Conway’s Law We will learn how to build perfectly functioning software

University of Southern California Center for Software Engineering C S E USC ©USC-CSE7 Converse of Conway’s Law We will learn how to build perfectly functioning software As soon as we learn how to build perfectly functioning organizations

University of Southern California Center for Software Engineering C S E USC ©USC-CSE8 The Lead Bullet Expectation If a lead bullet can kill a software-problem wolf this year, It will be able to do it next year too Counterexamples –Waterfall model of the software process –Pre-WYSIWYG word processing architecture –Pre-Web book sales management applications Key drivers: technology, economics, humanization

University of Southern California Center for Software Engineering C S E USC ©USC-CSE9 Information Technology Trends Traditional Development Standalone systems Stable requirements Rqts. determine capabilities Control over evolution Enough time to keep stable Value-insensitive process models Current/Future Trends Everything connected Rapid requirements change COTS capabilities determine rqts. No control over COTS evolution Ever-decreasing cycle times Value-oriented process models

University of Southern California Center for Software Engineering C S E USC ©USC-CSE10 Lead Bullets with Dwindling Niches Complete, consistent, traceable, testable snapshot requirements Static domain architectures and enterprise architectures Heavyweight formal methods Fixed contract models of software management COTS- and value-insensitive object- oriented methods

University of Southern California Center for Software Engineering C S E USC ©USC-CSE11 Core Niches for Current Lead Bullets - Still very important High-assurance, Real-time, Autonomous control systems

University of Southern California Center for Software Engineering C S E USC ©USC-CSE12 Underlying World-View Trends Universalism  Situationalism –Stephen Toulmin, Cosmopolis, U. Chicago Press, 1990 Reductionism  Emergence –Stuart Kauffman, At Home in the Universe, Oxford U. Press, –W. Brian Arthur, “Increasing Returns and the New World of Business,” Harvard Business Review, Jul/Aug 1996, pp –James Highsmith, Adaptive Software Development, Dorset House, Software Focus  System Focus –John Thorp, The Information Paradox, M c Graw Hill, 1998.

University of Southern California Center for Software Engineering C S E USC ©USC-CSE13 Cosmopolis: The Erosion of Modernist Philosophy Dominant since 17 th century Formal, reductionist –Apply laws of cosmos to human polis Focus on written vs. oral; universal vs. particular; general vs. local; timeless vs. timely –one-size-fits-all (lead bullet) solutions Strong influence on focus of computer science Weak in dealing with human behavior, rapid change

University of Southern California Center for Software Engineering C S E USC ©USC-CSE14 Reductionism vs. Emergence Order is not imposed on complex adaptive systems; it emerges (Kauffman) Knowledge-based industries have increasing vs. decreasing returns (Arthur) –Network effects, up-front costs, customer groove-in –Adaptation succeeds better than optimization Adaptive model best fits future software projects (Highsmith) –Balance of discipline and flexibility

University of Southern California Center for Software Engineering C S E USC ©USC-CSE15 Outline Silver bullets and lead bullets Information technology trends –The dwindling lead-bullet niche –Underlying world-view trends Agile methods –Example: eXtreme Programming (XP) –Counterpoint: A skeptical view Relations to CMM and CMMI –How much planning is enough?

University of Southern California Center for Software Engineering C S E USC ©USC-CSE16 The Agile Manifesto - I Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: That is, while there is value in the items on the right, we value the items on the left more.

University of Southern California Center for Software Engineering C S E USC ©USC-CSE17 The Agile Manifesto – II Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Business people and developers must work together daily throughout the project.

University of Southern California Center for Software Engineering C S E USC ©USC-CSE18 The Agile Manifesto – III Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Working software is the primary measure of progress. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

University of Southern California Center for Software Engineering C S E USC ©USC-CSE19 The Agile Manifesto – IV Continuous attention to technical excellence and good design enhances agility. Simplicity – the art of maximizing the amount of work not done – is essential. The best architectures, requirements, and designs emerge from self-organizing teams. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

University of Southern California Center for Software Engineering C S E USC ©USC-CSE20 Various Agile Methods Available Adaptive Software Development (ASD) Agile Modeling Crystal methods Dynamic System Development Methodology (DSDM) * eXtreme Programming (XP) Feature Driven Development Lean Development Scrum

University of Southern California Center for Software Engineering C S E USC ©USC-CSE21 eXtreme Programming (XP) Principles The 12 Practices Early Adopters

University of Southern California Center for Software Engineering C S E USC ©USC-CSE22 XP Principles – I Philosophy: Take known good practices and push them to extremes “If code reviews are good, we’ll review code all the time” “If testing is good, we’ll test all the time” “If design is good, we’ll make it part of everybody’s daily business”

University of Southern California Center for Software Engineering C S E USC ©USC-CSE23 XP Principles – II “If simplicity is good, we’ll always leave the system with the simplest design that supports its current functionality” “If architecture is important, everybody will work defining and refining the architecture all the time” “If integration testing is important, then we’ll integrate and test several times a day”

University of Southern California Center for Software Engineering C S E USC ©USC-CSE24 XP Principles – III “If short iterations are good, we’ll make the iterations really, really short – seconds and minutes and hours, not weeks and months and years” “If customer involvement is good, we’ll make them full-time participants”

University of Southern California Center for Software Engineering C S E USC ©USC-CSE25 XP: The 12 Practices The Planning Game Small Releases Metaphor Simple Design Testing Refactoring Pair Programming Collective Ownership Continuous Integration 40-hour Week On-site Customer Coding Standards -Used generatively, not imperatively

University of Southern California Center for Software Engineering C S E USC ©USC-CSE26 The Planning Game Use stories to facilitate knowledge transfer Put decisions in the hands of the person with the best knowledge: –business decisions Customer –software decisions Developer Plan only as far as your knowledge allows –next iteration or next release

University of Southern California Center for Software Engineering C S E USC ©USC-CSE27 Small Releases Supports quick feedback from users Simplify the tracking of metrics –stories per iteration project velocity Increase the manageability of the project for the customer –But complicate user conservation of familiarity

University of Southern California Center for Software Engineering C S E USC ©USC-CSE28 Metaphor Ground all discussions in a single shared story of how the whole system works Provide an overarching view of the project Connect program to work process

University of Southern California Center for Software Engineering C S E USC ©USC-CSE29 Simple Design Design Embodies only the needed complexity and no more –emphasis on top-down or bottom-up design as needed to meet this iteration’s stories –extra complexity removed when discovered Simpler designs are easier to modify, maintain, and describe –decreases the cost of design changes –But no notion of product line architecture

University of Southern California Center for Software Engineering C S E USC ©USC-CSE30 Testing Unit tests verify the programmer’s work –must be done by programmer –constant testing makes finding new bugs faster and easier Functional tests verify that a story is complete –developed by customer –tests define functional requirements

University of Southern California Center for Software Engineering C S E USC ©USC-CSE31 Refactoring Procedure for implementing iterative design –behavior-preserving –improves communication among developers –adds flexibility to the programming process Design is important – do it all the time –software development process is a design process –But redesign much more expensive for large systems

University of Southern California Center for Software Engineering C S E USC ©USC-CSE32 Pair Programming All code is written by two programmers at a single machine Inspections are important, so do them all the time Increase implicit knowledge transfer Decrease cycle time, but increase effort

University of Southern California Center for Software Engineering C S E USC ©USC-CSE33 Collective Ownership Everyone owns all of the code –anyone can change any code anywhere –no personal ownership of modules –no egoless programming either Everyone is permitted access to all the code so everyone has a stake in knowing all of the code (that they will work with) Requires deserved trust –But still has scalability problems

University of Southern California Center for Software Engineering C S E USC ©USC-CSE34 Continuous Integration The system always works –there is always something to be released Similar to rapid releases –fast feedback to developers on problems –no ‘big bang’ integration disasters

University of Southern California Center for Software Engineering C S E USC ©USC-CSE35 40-hour Week No heroes Knowledge can only be transferred at a limited rate Work for sustained speed, not a single sprint –never work overtime a second week in a row

University of Southern California Center for Software Engineering C S E USC ©USC-CSE36 On-site Customer A real, live user available full-time to answer questions as they occur Programmers don’t know everything Business knowledge is the key to a successful business project

University of Southern California Center for Software Engineering C S E USC ©USC-CSE37 Coding Standards Communication occurs through the code Common standard promotes understanding of other developers’ code Helps promote team focus

University of Southern California Center for Software Engineering C S E USC ©USC-CSE38 Counterpoint: A Skeptical View – I Letter to Computer, Steven Rakitin, Dec “individuals and interactions over processes and tools” Translation: Talking to people gives us the flexibility to do whatever we want in whatever way we want to do it. Of course, it’s understood that we know what you want - even if you don't. “working software over comprehensive documentation” Translation: We want to spend all our time coding. Real programmers don’t write documentation.

University of Southern California Center for Software Engineering C S E USC ©USC-CSE39 Counterpoint: A Skeptical View – II Letter to Computer, Steven Rakitin, Dec “customer collaboration over contract negotiation” Translation: Let's not spend time haggling over the details, it only interferes with our ability to spend all our time coding. We’ll work out the kinks once we deliver something... “responding to change over following a plan” Translation: Following a plan implies we would have to spend time thinking about the problem and how we might actually solve it. Why would we want to do that when we could be coding?

University of Southern California Center for Software Engineering C S E USC ©USC-CSE40 Outline Silver bullets and lead bullets Information technology trends –The dwindling lead-bullet niche –Underlying world-view trends Agile methods –Example: eXtreme Programming (XP) –Counterpoint: A skeptical view Relations to Software CMM and CMMI –The planning spectrum –Agile and Plan-Driven home grounds –How much planning is enough?

University of Southern California Center for Software Engineering C S E USC ©USC-CSE41 The Planning Spectrum Hackers XP Adaptive Sw Devel. Milestone Risk- Driven Models …… Milestone Plan-Driven Models Inch- Pebble Ironbound Contract Software CMM Agile Methods CMMI

University of Southern California Center for Software Engineering C S E USC ©USC-CSE42 Agile and Plan-Driven Home Grounds Plan-oriented developers; mix of skills Mix of customer capability levels requirements knowable early; largely stable Architected for current and foreseeable requirements Refactoring expensive Larger teams, products Premium on high-assurance Agile, knowledgeable, collaborative developers Dedicated, knowledgeable, collaborative, representative, empowered customers Largely emergent requirements, rapid change Architected for current requirements Refactoring inexpensive Smaller teams, products Premium on rapid value Agile Home GroundPlan-Driven Home Ground

University of Southern California Center for Software Engineering C S E USC ©USC-CSE43 How Much Planning Is Enough? - A risk analysis approach Risk Exposure RE = Prob (Loss) * Size (Loss) –“Loss” – financial; reputation; future prospects, … For multiple sources of loss: sources RE =  [Prob (Loss) * Size (Loss)] source

University of Southern California Center for Software Engineering C S E USC ©USC-CSE44 Example RE Profile: Planning Detail - Loss due to inadequate plans Time and Effort Invested in plans RE = P(L) * S(L) high P(L): inadequate plans high S(L): major problems (oversights, delays, rework) low P(L): thorough plans low S(L): minor problems

University of Southern California Center for Software Engineering C S E USC ©USC-CSE45 Example RE Profile: Planning Detail - Loss due to inadequate plans - Loss due to market share erosion Time and Effort Invested in Plans RE = P(L) * S(L) low P(L): few plan delays low S(L): early value capture high P(L): plan breakage, delay high S(L): value capture delays high P(L): inadequate plans high S(L): major problems (oversights, delays, rework)) low P(L): thorough plans low S(L): minor problems

University of Southern California Center for Software Engineering C S E USC ©USC-CSE46 Example RE Profile: Time to Ship - Sum of Risk Exposures Time and Effort Invested in Plans RE = P(L) * S(L) low P(L): few plan delays low S(L): early value capture high P(L): plan breakage, delay high S(L): value capture delays Sweet Spot high P(L): inadequate plans high S(L): major problems (oversights, delays, rework) low P(L): thorough plans low S(L): minor problems

University of Southern California Center for Software Engineering C S E USC ©USC-CSE47 Comparative RE Profile: Plan-Driven Home Ground Time and Effort Invested in Plans RE = P(L) * S(L) Mainstream Sweet Spot Higher S(L): large system rework Plan-Driven Sweet Spot

University of Southern California Center for Software Engineering C S E USC ©USC-CSE48 Comparative RE Profile: Agile Home Ground Time and Effort Invested in Plans RE = P(L) * S(L) Mainstream Sweet Spot Lower S(L): easy rework Agile Sweet Spot