James A. Whittaker Principal Architect Visual Studio Team Test Microsoft

“A mathematician is a device that turns coffee into theorems” “A computer programmer is a device that turns lattes into bugs”

I knew less than 10 people with addresses No one ever asked me to fix their computer I never had to reboot toys, phones, picture frames, tools, dog collars, cable boxes, …

By the time you make breakfast … (Software is in your coffee maker and microwave) … and drive to work … (It’s in your automobile and the traffic lights) … and make it to your office … (It’s in the keyless entry system and elevators) …millions of lines of code have executed on your behalf

Consider that your … – Finances, credit and tax information – Travel documents and records – Traffic violations and criminal history – Citizenship, travel history and visa information – Medical records – Organization memberships … are all stored in and processed by software

Your personal information is stored on, literally, hundreds or thousands of computers all over the world Let’s not forget that: – Your electricity, food, water and other necessities are provided at the mercy of automated dispatch and delivery mechanisms – Nuclear power and advanced weapons are controlled by software – Airplanes take off and are guided in flight by software – The people working on curing cancer, countering climate change, disease control, finding killer asteroids are all depending on software

Software has a less-than-stellar reputation for quality As users, how many of you have been inconvenienced, annoyed or totally hosed by software? What percentage of users have experienced the same?Is it any wonder our industry calls them “users”?

In the future software will become even more pervasive What will happen when these systems fail? How can we create reliable software and what role does the tester play? We need to solve some very fundamental problems

Too many cooks/too many side dishes What does software look like anyway? Too much distance between creation and detection

Too many cooks/too many side dishes

Architects see Visio diagrams and flow charts Program Managers see PowerPoint decks and Word storyboards Developers see Visual Studio Testers see binaries and interfaces

Too many cooks/too many side dishes Testers see binaries and interfaces

Too many cooks/too many side dishes Testers see binaries and interfaces What the user sees What the tester sees

Too many cooks/too many side dishes From where I sit: – All these side dishes need to make a good meal (software) – The architecture, design, code … they are all useful to testers in the right context I want to hover over a UI element and see code, data types, value ranges, previous bugs, test history … ‘Cheating’ is not wrong when you are trying to build a great product!

Well, there’s: – Input – Output – Data flow – Control flow – Modules – Dependencies – Environment variables – Files – Interfaces What does software look like anyway?

From where I sit: – The inability to visualize our products and their intricacies makes testing software artificially complex – We need to be able to see the product as it is being built and as it executes – For testers this means leaving Lewis and Clark behind and modernizing our tools What does software look like anyway?

Too much distance between creation and detection

Spell checkers in the ‘old days’ vs. spell checkers now – Spell checking and grammar checking is instantaneous – Huge decrease in copy edit and review cycle What about static analysis and testing? – Why wait until the program is complete? – Right-click  TestMeNow in the IDE Too much distance between creation and detection

From where I sit: – The time between bug creation and bug detection is a key factor in long ship cycles – We need to find bugs at the absolute earliest possible moment – For testers this means pushing test techniques and tools into early lifecycle phases Too much distance between creation and detection

Software will continue to run our economy, our businesses and our lives Software will be part of the solutions that will grow food, cure disease and produce sustainable energy Remove software from the equation and humankind’s problems get a lot harder It is imperative that we get software right!

20 years from now – Will the quality of software be taken for granted? Will users be genuinely surprised when it fails? – Will researchers look back in wonder that there was ever even a need for conferences like this? – What hard problems will simply cease to be hard because the software industry of this time did what needed to be done?

A time of great invention and opportunity is upon us The future of software development is waiting to be invented The world desperately needs it

Slides available later today at: