Moving to Agile in an FDA Environment An Experience Report August 27, 2009

Introduction Available for download at http://agile2009.agilealliance.org/ Agile Resources Agile Manifesto http://www.agilemanifesto.org/ Agile Alliance http://www.agilealliance.org Agile Books http://www.agiletek.com/thoughtleadership/books “Agile Software Development” by Alistair Cockburn “Agile Project Management” by Jim Highsmith Presenter: Tim Questions for audience How many of you have experience using agile? How many of you have experience working on medical devices? How many of you have experience using agile on medical device projects? Any audience members from the FDA? Brief AgileTek company overview Presenter: Rod Brief Abbott company overview

Outline The Abbott Experience Lessons Learned Results The least burdensome approach? A Risk Based Approach Control what you don’t know, don’t let it control you Dispense with ceremony Results Presenter: Rod

The Abbott Experience Comparing two FDA-regulated medical device projects One not agile, one agile Class III devices (most stringent) Results of agile adoption Lower cost Shorter duration Better, less prescriptive test cases Accommodated change Higher quality See paper “Adopting Agile in an FDA Regulated Environment” Presenter: Rod Describe Architect and m2000 Describe software process used for both products Talk about evolution of agile at Abbott Talk about staffing and inherent inefficiencies in the process

Moving to Agile Convincing Management Test drive Change happens Will vary widely depending on corporate culture Our experience: Hit targets Produce regulatory artifacts Engage quality organization early Describe mapping of ‘agile’ artifacts to traditional artifacts Test drive While still in Concept or Feasibility. Late development or support probably NOT a good strategy for success. Change happens Get used to it Deal with it Get over it Plan for ‘face time’ Iteration planning minimum Ideal is co-location Not always feasible Presenter: Rod - One strategy: use existing process with ruthlessly narrow scope. - Pick a few features and do them Aim for short scope 4 to six weeks - Better still do them independently and in parallel - Integrate - Rinse and Repeat - EMPHASIS: DON’T NEED ALL REQUIREMENTS UPFRONT - Stayed in feasibility for about ½ of m2000 project - Evolving artifacts

Software Development Lifecycle Presenter: Rod - Describe the 3 C’s in more detail

Iteration Model Presenter: Rod - Describe how work products were allocated to phases - Earlier iterations had more emphasis on characterization, planning, product definition - Indicate that artifacts map to regulatory requirements Question: Should we include an example of a regulatory requirement and how we satisfied it?

Sample Design Control Documents Presenter: John Build a matrix of artifacts Living documents Monitor progress at every iteration planning session Treat with same importance as code and test cases The final version of these documents will comprise most of the technical portion of your submission Courtesy of: Certified Compliance Solutions, Inc.

Lessons Learned

Least Burdensome We [FDA] are defining the term “least burdensome” as a successful means of addressing a premarket issue that involves the most appropriate investment of time, effort, and resources on the part of industry and FDA. --The Least Burdensome Provisions of the FDA Modernization Act of 1997: Concept and Principles; Final Guidance for FDA and Industry Presenter: John - Lead off with the “concern that lesser documentation will not be acceptable with the FDA” - This terminology is pervasive in the FDA’s guidance documentation - Testing Tendency to write overly detailed tests One small change and poof all your tests are wrong and MUST be rerun (due to change) Be Less Prescriptive - KISS

A Risk Based Approach Courtesy of: Certified Compliance Solutions, Inc.

FDA General Principles of Software Validation, page 7, section 3.1.2: Risk Based Approaches IEC 62304 section 5.1.1, page 31: Note 1: The software model can identify different activities for different software items according to the safety classification FDA General Principles of Software Validation, page 7, section 3.1.2: The level of confidence, and therefore the level of software validation, verification, and testing effort needed, will vary depending on the safety risk (hazard) posed by the automated functions of the device. Presenter: John - Risk Focus Not all risks are equal Emphasis on high risks and mitigations Impacts requirement details and testing effort - Focuses design on critical areas - Provides a basis for defining different levels of test and review - Increasingly emphasized by FDA and ISO/IEC - Integrates external standards compliance (IEC, UL, etc.) Courtesy of: Certified Compliance Solutions, Inc.

FDA Reviewer Guidance 2005 Presenter: John - Plan on spending a disproportionate amount of time/detail on testing the requirements/features that are safety related (e.g. results calculation that impacts patient treatment) Minor: No injury or damage to health is possible (e.g. tongue depressor) Moderate: Non-serious injury is possible (e.g. clinical chemistry medical device) Major: Death or serious injury is possible (e.g. dialysis instrument) - PMA Guidance for the content of pre-market submissions for software contained in medical devices Courtesy of: Certified Compliance Solutions, Inc.

Control What You Don’t Know, Don’t Let It Control You What you do know: A typical medical device is developed over a 3-5 year time horizon It is a myth that you can predict in detail your end product requirements up-front Start with a core set of features that you must implement Implement the core features first Defer the most volatile features as long as possible Iterative approach allows the team to: Manage scope and limit feature creep Negotiate scope and tradeoffs with key stakeholders “At time of commercial launch, a number of features, once thought to be essential, were not included. Some were deferred as long as three years. Nonetheless, the product was considered highly successful and trading off nice to have features for three years of sales is an easy choice.” Presenter: Rod - Requirements Be Less Prescriptive Detailed requirements will simply be wrong quicker.

Dispense With Ceremony If it is not adding value, and it is not required, do not do it The design history files should contain the minimum set of documentation that satisfies the regulatory requirements There will be other activities that you will want to document, no need to include in design history file, make sure they add value and do it in a least burdensome way Avoid doing things because “that’s the way we’ve always done it” If it feels like you are wasting your time you probably are Presenter: Rod Wedding analogy justice of the peace vs. formal big wedding

Results

Results High visibility Lower cost and shorter duration Higher quality Easier to manage and control Far fewer surprises Lower cost and shorter duration Estimated schedule and team size reduction of 20%-30% Estimated cost savings of 35%-50% Higher quality Availability of working software facilitated continuous testing instead of back loaded V&V Resulted in fewer overall defects, especially at the end of the project Better work life balance and team morale Project death marches are rarer because the issues are surfaced as you go and are managed accordingly, not all saved up for the end of the project Presenter: Rod - Early Integration Find problems faster Process People Hardware Software (problems? Who me?) - Show me something working - Getting stakeholder feedback: “What were you thinking???” Pre-market rules will encourage use of good internal proxies. - Tradeoff analysis extensibility maintainability quality

Q & A

Thank You Tim Hughes J.R. Jenks Managing Partner Managing Partner thughes@agiletek.com jrjenks@agiletek.com 847-699-2260 847-699-2250 John Skach Senior Technology Architect jskach@agiletek.com 847-699-2264 Rod Rasmussen Director, Informatics & Software Systems Rodney.Rasmussen@abbott.com 847-938-3633

Back-Up Slides

Product Feature Backlog Release Planning Product Feature Backlog Speculating Process Iterations: 1 to 6 Weeks Release Plan Iteration 0 Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5 Task Task Task

Roadmap of Releases Every iteration should be deployable code Most agile teams work within a roadmap of milestone releases 1 2 3 4 5 A B C D E F G H I J K L M N O P

Sample Submission Documents 1. Compliance Strategy: Map required documents to submission checklist Courtesy of: Certified Compliance Solutions, Inc.