1. When to Ship: A Practical Approach to Determining Application Deployment Readiness
Peter Varhol Solutions Evangelist
© 2011 Seapine Software, Inc. All rights reserved.

2. Agenda Why we don’t know when to ship How we can know when to ship
Data, reports, and shipping
The best laid plans
Really knowing when to ship

3. About the Presenter Peter Varhol Evangelist at Seapine Software
Former professor of computer science and mathematics
Long-time speaker and writer on technology topics
Founder of the Cutting Edge Computing blog –

4. When the schedule tells us to
We All Know When to Ship
When the schedule tells us to
It’s not a bad metric, but it doesn’t take into account whether or not the application is really ready. That’s what we’re going to consider in this presentation.

5. Why We Don’t Know When to Ship
We haven’t defined shipping criteria at the beginning
We may not even agree on criteria
We don’t have good measurements of completeness and quality
We don’t have contingency plans when things go wrong

6. We All Know When to Ship
Therefore, the schedule isn’t a good metric for shipping
We don’t know if the software is ready
In fact,
What does “ready to ship” really mean?
And if we don’t know what it is, we can’t measure it

7. Why We Don’t Know When to Ship
The meaning of “shipping software” has changed
In the past
We did beta testing, candidate releases, manufacturing and packaging
Today
The process varies, but can be as short as a day
Fix, test, rebuild Web application

8. Knowing When to Ship
“The companies that know how to ship software are the ones to watch.” Mark Lucovsky Software Engineer, Google Former Distinguished Software Engineer, Micrososft

9. We All Know When to Ship Code completeness
Developers and testers determine code completeness
Quality
Testers and users determine actual quality
Business
There are important business decisions in shipping

10. We All Know When to Ship We need quality standards
And ultimately metrics to measure those standards
We need agreement on the standards
And confidence in our metrics
We may have to trade off
But we want a seat at that table

11. We All Know When to Ship Possible measurements
All features seem to be working properly
Failures aren’t replicable
We think it’s ready
There are a number of reasons that quality is a problem in many software development organizations.

12. We All Know When to Ship
But there are more objective indicators, focusing on quality
No P0 or P1 bugs
New bug count is asymptotic
All functional tests execute for 10 consecutive days

13. Requirements Play a Part
All requirements are satisfied
That may take a long time
All requirements are not created equal
Some are more important than others
Requirements change

14. We All Know When to Ship How about the business?
Balance of revenue and quality
Shipping can start a new revenue stream
But quality shortcomings could dent it

15. We All Know When to Ship Any definition is possible
As long as it meets the needs of the team and the business
And can be measured

16. We All Know When to Ship It depends on the application
Safety-critical versus commercial versus internal
A proposed quality definition
Full feature coverage, no high-priority bugs
A set code coverage percentage
Find/fix defect ratio low and stabilized

17. We All Know When to Ship

18. Define What It Means to Be Ready
Code complete
All required features implemented or accounted for
All error-handling code has been implemented
Unit tests have been run

19. Define What It Means to Be Ready
Business considerations
We need the revenue this quarter
We’ve promised our customers or investors
We have a limited market window

20. Define What It Means to Be Ready
Quality
Base quality goals on projected operational use
Quantify those goals and get agreement with stakeholders
Determine how to test and measure to that level of quality

21. Implement a Solution A dashboard can work well Define quality measures
Determine dashboard representation
Enable drilldown
Generate reports on status and trends

22. Quantifying Ready to Ship
You need
A platform for data collection
The ability to analyze data and create ongoing reports
The ability to do ad hoc queries
The ability to share reports and data

23. Quantifying Ready to Ship
Measuring feature coverage
Organize test cases
Track test execution
Gather and analyze statistics

24. Quantifying Ready to Ship
Code coverage percentage
A developer’s metric
Close to 100 percent is unrealistic

25. Measuring Ready to Ship

26. Measuring Ready to Ship

27. The Best Laid Plans of Mice and Men
You had an agreement on quality and its measures
Something goes wrong
Development takes too long
The project scope or direction changes
What do testers do now?

28. The Best Laid Plans of Mice and Men
Quality now has a seat at the table
It’s quantified and agreed to ahead of time
You may still have to compromise
It still has to ship this quarter
But you communicate the impact on quality

29. The Best Laid Plans The result could be An extended schedule
If the schedule remains the same
Fewer features in this release
Follow-up with a service pack
Explanations to customers/users

30. Really Knowing When to Ship
A quantifiable definition is essential
A number of possibilities, depending on the application
Measurements are ongoing
And must be analyzed in the context of the project
Know what it means to ship
Features, code, defects

31. Summary Define your criteria for shipping
Determine information you need on that criteria
Prepare reports and live charts
Now you know when to ship

32. Questions?

33. Thank you
© 2011 Seapine Software, Inc. All rights reserved.