1. March 30, 2004

2. Exploratory Testing

3. Testing Approaches Analytical Information-driven Intuitive Exploratory Design the tests and test concurrently Learn the system and test it as you go Structure creative testing Think while testing Risk-based Analogous to Extreme Programming

4. Exploratory Testing Tasks Explore Elements of the product How the product should work Design Tests Which elements Speculate on possible quality problems Execute Tests Observe behavior Evaluate against expectations All with test design techniques best suited for the product

5. Exploratory Testing Practice Used to probe for weak areas Especially useful when Weak specifications and requirements Little domain knowledge Time pressures Less appropriate when Well-defined test requirements Strong need for regression testing Repeatable over releases Cost of maintenance Few new test cases

6. Planning Decompose the product into elements Areas of function that you can test in 1-2 days Define charters Decomposition into units that can be tested in 1-2 hours Quality criteria Capability, reliability, usability, performance, installability, compatibility, … Select test techniques

7. Charter Provides clear mission of why this test Suggests what and how it should be tested, as well as problems to look for Not a detailed plan, but should be as specific as possible Might include risks, documents and desired output

8. Maintenance

9. Cost of Maintenance Estimates of percentage of total life cycle cost: 40% - 90%

10. Software Maintenance Types Adaptive maintenance: changes needed as a consequence of operation system, hardware, or DBMS changes Corrective maintenance: the identification and removal of faults in the software Perfective maintenance: changes required as a result of user requests Preventive maintenance: changes made to software to make it more maintainable

11. Why adaptation? Lehman’s Law: if a program doesn’t adapt, it becomes increasingly useless Example: programs that didn’t adapt to the web The majority of maintenance is concerned with evolution deriving from user requested changes.

12. Lehman’s Second Law As an evolving program changes, its structure tends to become more complex. Extra resources must be devoted to preserving the semantic and simplifying the structure. For most software, nothing has been done about it, so changes are increasingly more expensive and difficult.

13. How is Maintenance Initiated? Most common: user request Types of requests Bug reports Accepted Working as designed Documentation fix Feature requests New environments

14. Steps for handling a change Understand the problem Design the changes Analyze impact Implement changes Update documentation Regression test Release

15. Patches What is a patch? Quick fix that doesn’t go through the full process When should it be used? Error that is preventing use of the system Problems with use Multiple patches can be order dependent Users can barely track which ones have been applied Code version explosion Permanent fix may or may not be compatible

16. Problems of Maintenance Organizational Alignment with objectives Cost benefit analysis Process Impact Documentation Regression testing Technical Building software that is maintainable

17. Different Objectives over Time At release: bug-free Six months later: competitive or competition-leading features Two years later: reduce maintenance cost

18. Outsourcing of Maintenance Outsourcing: inside or outside the company Advantages Less expensive Uses less skilled people Frees developers for the next release Disadvantages Changes can be slower May still require developer help Code can become less maintainable

19. Cost Benefit Analysis (Risk Analysis) Will this problem reduce the number of programs that I sell? Will this problem impact future sales/ How many people will it affect? How important are the customers it will affect? Is it a “show stopper” or an annoyance?

20. Process Issues Impact analysis Updating documentation Reengineering Legacy systems Reverse engineering

21. Impact analysis How much of the system will it impact? How complex are the changes? Heuristics: Percentage of requirements that are impacted Where was the defect injected (requirements, design, code)

22. Updating documentation A requirement defect can impact Requirements Use cases Architecture Design Code Where in the process do you update? No matter when, there is a period of inconsistent documents!

23. Reengineering Code gets messy over time Extreme programming re-factoring At some point, quality suffers Changes slow Fixes introducing errors Need to invest in the code! Rules as to when to rewrite a module Abstractions: variables -> methods Harder: when is REDESIGN needed?

24. Legacy Systems Existing systems that are still useful May not want to invest in enhancements Future functions will use new process May not be able to easily modify Unsupported language or libraries Lack of skills No source code available!

25. Handling Legacy Systems Incorporation Business as usual Encapsulation Accessed from new system Adapters Wrapper around the legacy system Adapters in new system

26. Technical Issues Building maintainable software What features have we talked about in code and documentation that help?

27. Technical Issues Building maintainable software What features have we talked about in code and documentation that help? Well documented code Names, headers, style, … All the documentation Architecture, design documentation, use cases, requirements, …

28. Today’s Tip You need to make sure that when you turn in your final project, everything is consistent. Contract can’t be changed Other documents

29. For your enjoyment… “Yes, the world needs bankers and artists and even professional athletes. They, among countless others, create the breadth of society and culture. But if you want tomorrow to come – if you want to spawn entire economic sectors that didn’t exist yesterday – those are not the people you turn to. It’s technologists who create that kind of future.” (Neil deGrasse Tyson, Director of Hayden Planetarium)