1. James A. Whittaker Principal Architect Visual Studio Team Test Microsoft blogs.msdn.com/james_whittaker

2. to err is HUMAN

3. nothing is PERFECT

4. WE all make MISTAKES

5. IT could have happened to ANYONE

6. We have many expressions to capture the fact that humans are fallible

7. Our creations and inventions echo this unnerving tendency

8. to FAIL

9. if there was an Olympics for FAILURE

10. Or a World Cup for that which sucks

11. surely SOFTWARE would reign supreme

14. Software … … stocks supermarket shelves … delivers electricity and water … stores our personal information … runs nuclear power plants … controls doomsday weapons …

15. Are these failures typical? How many of you have seen software fail? How many users share this same experience?

16. Any reason to think things will improve? Let’s take a look at the future and find outfuture

17. How will we test the applications of tomorrow?

18. Virtualization Information Visualization

19. Virtualization Information Visualization

20. Virtualization

21. Users are effective testers Let’s leverage their power – They have the environments we seek – They have the data we lack – They run the tests we don’t think of “Crowd-sourcing” is one answer I like Virtualization is the next step

22. Virtualization The story today: – A user (or tester) finds a bug, developer can’t repro it – A tester wants to reuse test cases, but can’t get them to run Virtualization solves this – User creates a VM at the point of failure These VMs have a market value Developers could have access to the sum total of all the environments in which their application can run – A tester creates VMs for every test suite We now have reusable, fully transferable testing assets An automated historical record of all testing on our app If we can store every video in the world, why not every test case/test data/user environment? – It’s not been possible before but it is now

23. Virtualization What does the virtualized testing world look like? – Think crowd-sourcing on steroids – Test VM’s have great value, businesses emerge owning and managing these environments – Testers analyze an app, license the appropriate Test VM’s, conduct the test and report results Companies no longer need QA departments Why test a subset of possible usage when (in the future) we can test them all? Software is important, it deserves this kind of coverage

24. Virtualization Information Visualization

25. Virtualization Information Visualization

26. Information

28. Information about me (Status, Health, etc.) Quick Access to oft- used abilities. Information about my target. Information about my location in the world.

29. Information What is the information we need? – Architects see Visio diagrams and flow charts – Program Managers see PowerPoint decks and Word storyboards – Developers see Visual Studio – Testers see binaries and interfaces In this model, testers see only what the user sees … I object!

30. Information What the user sees What the tester sees

31. Information Testers need more information – The architecture, design, code … they are all useful to testers in the right context – I want to hover over a UI element and see code, data types, value ranges, previous bugs, test history … – ‘Cheating’ is not wrong when you are trying to build a great product!

32. Virtualization Information Visualization

33. Virtualization Information Visualization

37. Well, there’s: – Input – Output – Data flow – Control flow – Modules – Dependencies – Environment variables – Files – Interfaces – Bugs Visualization

38. Software will continue to run our economy, our businesses and our lives Software will be a critical part of the solutions that will grow food, cure disease and produce sustainable energy Remove software from the equation and humankind’s problems get a lot harder It is imperative that we get software right!

39. 20 years from now – Software testing will look fundamentally different – The tools will be more powerful and allow us to be more impactful – Software testers will contribute more fundamentally to the development process

40. 20 years from now – Will the quality of software be taken for granted? Will users be genuinely surprised when it fails? – Will researchers look back in wonder that there was ever even a need for dedicated bug finders? – What hard problems will simply cease to be hard because the software industry of this time did what needed to be done?

41. James A. Whittaker, PhD Blog: blogs.msdn.com/james_whittaker Email: jw@microsoft.com URL: msdn.microsoft.com/testercenter