® IBM Software Group © IBM Corporation Agile Software Development: The Full Story Scott W. Ambler Practice Leader Agile Development

IBM Software Group | Rational software Scott Ambler - Background  Practice Leader Agile Development  Senior Contributing Editor, Dr. Dobb’s Journal  Fellow – International Association of Software Architects  

IBM Software Group | Rational software Agenda  Warning!  Agile Current Status  Common Agile Practices  Scaling Practices  Call to Action

IBM Software Group | Rational software Warning!  I’m spectacularly blunt at times  Many new ideas will be presented  Some may not fit well into your existing environment  Some will challenge your existing notions about software development  Some will confirm your unvoiced suspicions  Don’t make any “career-ending moves”  Be skeptical but open minded

IBM Software Group | Rational software Agenda  Warning!  Agile Current Status  Agile Adoption Rates  Project Success Rates  Common Agile Practices  Scaling Practices  Call to Action

IBM Software Group | Rational software Has Your Organization Adopted One or More Agile Techniques? 85% have run multiple agile projects 24% of “No” respondents hope to do Agile this year Source: Dr Dobb’s 2007 Agile Adoption Survey

IBM Software Group | Rational software % of Successful Agile Projects (296 co-located, 251 not co-location, 130 offshoring): Agile Adoption Survey

IBM Software Group | Rational software Largest Team Size Attempted vs. Successful

IBM Software Group | Rational software Why Agile/Lean? It’s More Successful  Quality: 87.3% believe that delivering high quality is more important than delivering on time and on budget  Scope: 87.3% believe that meeting actual needs of stakeholders is more important than building the system to specification  Money: 79.6% believe that providing the best ROI is more important than delivering under budget  Staff: 75.8% believe that having a healthy workplace is more important than delivering on time and on budget  Schedule: 61.3% believe that delivering when the system is ready to be shipped is more important than delivering on schedule Source: Dr Dobb’s 2007 Project Success Survey

IBM Software Group | Rational software Agenda  Warning!  Agile Current Status  Common Agile Practices  Agile Development Practices  Test-Driven Development (TDD)  Database Refactoring  Other Quality Practices  Working in Priority Order  Agile Planning  Agile Model Driven Development (AMDD)  Agile User Experience  Agile Documentation  Scaling Practices  Call to Action

IBM Software Group | Rational software Agile Development Practices  Regular Delivery of Working Software  Only valid measure of progress  Provides visible results to stakeholders  True earned value, not documentation-based “earned value”  Daily Stakeholder Interaction  On-Site Customer  Active Stakeholder Participation  Product Owner  Continuous Integration  Automatically compile, test, and style check your code  Continuous code integration is nice  Continuous system integration is nicer

IBM Software Group | Rational software Test First Design (TFD)  With TFD you write a single test and then just enough production code to fulfill that test  Test-Driven Development (TDD) = Refactoring + TFD  TDD is a just-in-time (JIT) specification activity  TDD is a continuous confirmatory validation activity  TDD via Customer/Acceptance Tests  Specification of requirements  TDD via Developer Tests  Specification of design  TDD is also called Behavior Driven Development (BDD)

IBM Software Group | Rational software Other Agile Quality Practices  Non-solo development  Pair programming  Modeling with others  Effectively continuous inspections  Following guidance  Coding practices  Database standards  User interface (UI) standards  Modeling style guidelines (  Refactoring  Small change to your code which improves the quality of the design without changing the semantics  Code refactoring  UI refactoring  Database refactoring

IBM Software Group | Rational software Database Refactoring  A database refactoring is a simple change to a database schema that improves its design while retaining both its behavioral and informational semantics. Examples: Move Column, Rename Table, and Replace Blob With Table.  A database schema includes both structural aspects such as table and view definitions as well as functional aspects such as stored procedures and triggers.  Important: Database refactorings are a subset of schema transformations, but they do not add functionality. 

IBM Software Group | Rational software Working in Priority Order: Agile Change Management

IBM Software Group | Rational software Agile Planning  Create and maintain a high-level Gantt chart indicating the iterations, milestones, and major dependencies  Plan each iteration in detail at the beginning of the iteration  Done by the team, not just the manager  The people best suited to plan the work are the people who are going to do the work  Consider planning poker,  DDJ’s 2007 Adoption survey, most valuable work products:  #5 was an iteration task list  #18 was a high-level Gantt chart  #19 (of 19) was a detailed Gantt chart

IBM Software Group | Rational software Agile Model Driven Development (AMDD)  Do just enough initial envisioning to understand the scope and technical direction  Model storm on a just-in-time basis to gather the details when you need them

IBM Software Group | Rational software Agile User Experience (UEX)  Observations:  User interface (UI) and usability issues are critical to the success of most systems  The UI is the system to the end user  Few developers have solid UEX skills, although many think they do  Advice:  Everyone should have some UEX training  Have someone with UEX expertise within your organization, and ensure that they pair regularly  Part of initial envisioning should address UEX issues  UEX issues will need to be addressed throughout development  Recognize that few of us are building the iPod, but when we tread into new territory we may need to do more up-front work than usual

IBM Software Group | Rational software Agile Documentation Practices  Maximize stakeholder ROI  Are treated as a requirement  Have a specific customer and facilitate the work efforts of that customer  Are concise  Fulfill a purpose  Describe information that is less likely to change  Describe “good things to know”  Are sufficiently accurate, consistent, and detailed – But aren’t perfect

IBM Software Group | Rational software Agenda  Warning!  Agile Current Status  Common Agile Practices  Scaling Practices  Challenges with Mainstream Agile  Scaling TDD via Agile Model Driven Development (AMDD)  Scaling TDD’s via Comprehensive Testing  Scaling On-Site Customer/Product Owner  Scaling via Rational Unified Process (RUP)  Portfolio Management  Enterprise Architecture  Agile Data Management  Lean Development Governance  Call to Action

IBM Software Group | Rational software Challenges with Agile in the Mainstream Agile Development Co-located Geographical distribution Global Compliance requirement Low risk Critical, Audited Application complexity Simple, single platform Complex, multi-platform Team size Under 10 developers 100’s of developers Organization distribution (outsourcing, partnerships) In-house Third party Degree of Governance Informal Formal Entrenched process, people, and policy MinimalSignificant

IBM Software Group | Rational software Scaling TDD: Agile Model Driven Development (AMDD)

IBM Software Group | Rational software Scaling TDD: Comprehensive Agile Testing January 2007 Dr. Dobb’s Magazine (

IBM Software Group | Rational software Scaling XP’s On-Site Customer and Scrum’s Product Owner  On-site customer is nice, so put them to work  Stakeholders can be active participants in modeling  Product owner is really a communication conduit between the team and stakeholders  Must have agile business analysis skills  PO gets the team access to the relevant stakeholders just in time  Negotiate, negotiate, negotiate  Dr. Dobb’s Journal, January 2008

IBM Software Group | Rational software Database Testing

IBM Software Group | Rational software The Generic Agile Lifecycle

IBM Software Group | Rational software Scaling via Rational Unified Process (RUP)  RUP socialized many of the concepts taken for granted by the Agile community  RUP is really a process framework, not a process  RUP can be as Agile, or non-Agile, as you want to make it  Many organizations struggled to implement RUP effectively  RUP:  Addresses the fully development lifecycle  Is risk-driven  Contains advice for most of the challenges currently faced by Agile  RUP done right is Agile, RUP done wrong is just plain wrong

IBM Software Group | Rational software Portfolio Management  Driven by the enterprise vision and regulatory restrictions  Focus on collaboration and enablement, not command and control  Manage enterprise risk  Understand the as-is “IT inventory”  Identify potential projects  Choose the highest value projects  Organize similar projects into programs  Steer existing development projects and programs  Manage services contracts  Work closely with project managers  Monitor projects

IBM Software Group | Rational software Enterprise Architecture  Provide technical vision to the enterprise  Promote reuse and common infrastructure  Develop reference architectures  Develop guidance  Work closely with development teams  enterpriseArchitecture.html

IBM Software Group | Rational software Agile Data Management  Traditional data management has clearly failed:  Data Warehouse Institute (DWI) estimates data quality to have a $611B annual impact in the US  DDJ found that 62% of organizations have production data quality problems yet the majority have no viable strategy for addressing them  DDJ found that the majority of organizations have no database testing strategy in place, and many haven’t even considered it  DDJ found that over 60% of development teams now go around their organizations’ data groups  This is now the “elephant in the room” for most organizations  A new vision:  Evolutionary and collaborative approaches  Test-driven approaches  Dovetail into enterprise architecture and administration efforts, no longer a silo effort

IBM Software Group | Rational software Organization Roles & Responsibilities Processes Measures Policies & Standards Mission & Principles Lean Development Governance  Pragmatic Governance Body  Staged Program Delivery  Business-Driven Project Pipeline  Scenario-Driven Development  Simple And Relevant Metrics  Continuous Project Monitoring  Iterative Development  Adapt The Process  Risk-Based Milestones  Continuous Improvement  Embedded Compliance  Align HR Policies With IT Values  Align Stakeholder Policies With IT Values  Promote Self-Organizing Teams  Align Team Structure With Architecture  Integrated Lifecycle Environment  Valued Corporate Assets  Flexible Architectures

IBM Software Group | Rational software Agenda  Warning!  Agile Current Status  Common Agile Practices  Scaling Practices  Call to Action

IBM Software Group | Rational software A Call To Action  Look beyond development trees to see the business forest  Recognize that “the age of hype” is over  Talk about everything that we do, not just the cool/extreme things that we like to talk about  Bring agile concepts to other communities  Their questions will reveal many of the challenges we still face  Invite outsiders into our community  We need more “uncomfortable” keynotes  Police mailing lists a bit better  We turn off a lot of smart people who have something to contribute

® IBM Software Group © IBM Corporation Keep In Touch! Scott W. Ambler

IBM Software Group | Rational software References and Recommended Reading       Ambler, S.W. (2002). Agile Modeling: Effective Practices for XP and the UP. New York: John Wiley & Sons.  Ambler, S.W. (2003). Agile Database Techniques. New York: John Wiley & Sons.  Ambler, S.W. (2004). The Object Primer 3 rd Edition: AMDD with UML 2. New York: Cambridge University Press.  Ambler, S.W. and Sadalage, P.J. (2006). Refactoring Databases: Evolutionary Database Design. Reading, MA: Addison Wesley Longman, Inc.  Larman, C. (2004). Agile and Iterative Development: A Manager’s Guide. Reading, MA: Addison Wesley  McGovern, J., Ambler, S.W., Stevens, M., Linn, J., Sharan, V., & Jo, E. (2003). The Practical Guide to Enterprise Architecture. Prentice Hall PTR.

IBM Software Group | Rational software Agile Values We value: 1.Individuals and interactions 2.Working software 3.Customer collaboration 4.Responding to change Over: 1.Processes and tools 2.Comprehensive documentation 3.Contract negotiation 4.Following a plan

IBM Software Group | Rational software Agile Principles 1.Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. 2.Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. 3.Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. 4.Business people and developers must work together daily throughout the project. 5.Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. 6.The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. 7.Working software is the primary measure of progress. 8.Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. 9.Continuous attention to technical excellence and good design enhances agility. 10.Simplicity--the art of maximizing the amount of work not done--is essential. 11.The best architectures, requirements, and designs emerge from self-organizing teams. 12.At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.