1. Software Testing and Maintenance How the Web Brought Evolution Back Into Design
Jeff Offutt
SWE 437
George Mason University
Based on a keynote talk at SBES 2013

2. Some Engineering Historical Perspective
Building new technology incurs several costs
In this talk I will separate costs into four areas
Design
Production
Distribution
Support
Over time, the relative amount of these costs have continuously changed
We started with the ability to evolve our designs slowly
SWE 437
© Jeff Offutt

3. Pre-1850 : Hand-crafting
Design evolved over time, each new object better than the last
Low design costs
Very high production cost—weeks of labor
Low distribution cost—customers walked into the shop
Little or no support cost
SWE 437
© Jeff Offutt

4. 1850s : Assembly Line Manufacturing started to change this equation
The same design quickly put into thousands of products
Very low production cost
Much higher design costs
Distribution costs started to increase
Support costs increased—but were outsourced
SWE 437
© Jeff Offutt

5. 1900s: Automated Manufacturing
Robots increased the speed and efficiency of production
Production cost continued to decline
Instead of training people, design costs now included creating expensive robots
Distribution costs continued to increase
SWE 437
© Jeff Offutt

6. Post-WWII Worldwide Distribution
Distribution capabilities increased exponentially
Production costs continued to decline
Design costs continued to escalate
Support started to become “replace”
SWE 437
© Jeff Offutt

7. Manufacturing defeated evolutionary design !
2000s : Free Trade
This process has continued …
Free trade agreements
Cheap oil
Decreases in shipping costs
Decreases in production costs
The ultimate effect ?
Manufacturing defeated evolutionary design !
We now emphasize quantity over quality
SWE 437
© Jeff Offutt

8. Losing Evolutionary Design
Thousands of products are incredibly cheap
Many products are very low quality
Designed to last a few months or years, instead of decades
Instead of evolution, we have
Maintenance
Or replacement
But we lost something wonderful
Craftsmanship
SWE 437
© Jeff Offutt

9. Where is Software Engineering?
Ummm ... Excuse me, Professor ...
What does this have to do with SOFTWARE ENGINEERING ???
SWE 437
© Jeff Offutt

10. Traditional Software Development
Production cost for software is very low
Distribution cost is substantial
Includes marketing, sales, shipping
Support costs escalated
Software splits design into design and implementation
Both very expensive !
Instead of one design for each artifact, software has one design for many artifacts
SWE 437
© Jeff Offutt

11. Software evolution was very slow!
1900s Software Costs
Millions of customers skewed costs to the back end
High support costs
High distribution costs
New versions shipped every 4 to 6 years
MS Office, CAD, compilers, operating systems
Software needed to be “perfect out of the box”
Very expensive design
Very expensive implementation—including testing more than 50% of the cost
Software evolution was very slow!
SWE 437
© Jeff Offutt

12. Research Agenda
The need to be “perfect out of the box” heavily influenced decades of SE research
Formal methods
Modeling the entire system at once
Process
Testing finished products
Maintenance in terms of years
Much of our research focus and results assume :
High support costs
High distribution costs
High design costs
High implementation costs
SWE 437
© Jeff Offutt

13. Distribution Costs
In the 1980s, technology started driving down the distribution costs for software …
SWE 437
© Jeff Offutt

14. Then the World Wide Web changed everything
Usability and Support
As usability started to increase …
The need for support decreased
Then the World Wide Web changed everything
SWE 437
© Jeff Offutt

15. The web created a new way to deploy and distribute software
2000s : The Web
The web rearranged the importance of quality criteria, including making usability and reliability crucial
(1)
The web created a new way to deploy and distribute software
(2)
SWE 437
© Jeff Offutt

16. Traditional software deployment methods
Web Deployment
Traditional software deployment methods
Bundle
Shrink-wrap
Embed
Contract
5. Web Deployment
SWE 437
© Jeff Offutt

17. Distributing Web Software
Desktop software can be distributed across the web
Zero-cost distribution
Instantaneous distribution
This allows more frequent updates
Web applications are not distributed at all in any meaningful sense
Software resides on the servers
Updates can be made weekly … daily … hourly … continuously!
Mobile applications allow the artisan to come into your “home” to improve that rocking chair
SWE 437
© Jeff Offutt

18. This let us resuscitate evolutionary design !
Near-zero production costs …
Immediate distribution ….
Near-zero support costs …
This let us resuscitate evolutionary design !
SWE 437
© Jeff Offutt

19. Evolutionary Software Design
Pre-Web software design & production
Strived for a perfect design, expensive development
Deployed a new version ever 4 to 6 years
Evolution was very slow
Post-Web software production
Initial “pretty good” design and development
Slowly make it bigger and better
Faster evolution
Immediate changes to web applications
Automatic updates of desktop applications
Software upgrades pushed out to mobile devices hourly
Replacing chips in cars during oil changes
This changes all of software engineering !!
SWE 437
© Jeff Offutt

20. Industry Impacts How often is Google mail updated?
Daily … sometimes hourly
Piazza class support system
The first day I used it I reported a bug
It was fixed before I met class that afternoon
Sarah Allen invented youtube
She advises people with 5-year ideas to think about how they can achieve 1 idea in 6 months, and grow to the 5-year goal
SWE 437
© Jeff Offutt

21. Current Software Engineering
Software will not be designed and built
Software grows
Software needs to take responsibility for its own behavior
Waterfall is now, finally, thankfully, completely dead
Testing must focus on evolution, not new software
Yes, the web really does change EVERYTHING
SWE 437
© Jeff Offutt

22. How much do you know about agile processes ?
Software Process
We have already seen process changes that are a direct result of web deployment & distribution
Agile processes goals :
Have a working, preliminary, version as fast as possible
Continue growing the software to have more functionality and better behavior
Easy and fast to modify
Adapt to sudden and frequent changes in planned behavior
Agile processes are widely used
Results are mixed, but use is growing quickly
How much do you know about agile processes ?
SWE 437
© Jeff Offutt

23. Your generation needs to deal with this
Architecture
Software architects often assume their high level design will not change throughout development
And the lifetime of the system
It is not clear how this supports software growth, rapid deployment, and instantaneous distribution
Is this attitude compatible with agile processes ?
How does architecture design interact with refactoring ?
Your generation needs to deal with this
SWE 437
© Jeff Offutt

24. Software Self-Responsibility
Evolutionary design means we cannot know everything software will ever do
Self-management means the software adapts behavior to runtime changes—crucial for evolutionary design
Fault localization tries debug automatically, which can dramatically cut the human effort required to fix software after testing
Automated defect repair goes one step further, and attempts to automatically fix faults
Are you ready for the adaptive software revolution ?
SWE 437
© Jeff Offutt

25. TDD is an important part of this class
Software Testing
Test-driven design uses tests to drive requirements
Every step is evolutionary
We must stop thinking of regression testing as something special done “late in the process”
Virtually all testing is now regression testing
Model-based testing allows test design to quickly and easily adapt to changes
Test automation is the key to running tests as quickly as software is now changed
Model to implementation ?
Test oracle strategy ?
TDD is an important part of this class
SWE 437
© Jeff Offutt

26. Software Costs Then & Now
Old
New
Design : High
Design : Medium
Implementation : High
Implementation : Medium
Production : Low
Production : Zero
Distribution : High
Distribution : Zero
Support : High
Support : Low
SWE 437
© Jeff Offutt

27. Long Term Impact of Evolutionary Design
The end-result of large scale manufacturing was a heavy emphasis on quantity over quality
The web enables evolutionary design, which can allow us to focus on quality over quantity
What engineer would NOT love that ?
SWE 437
© Jeff Offutt