Trends in the Marketplace Testers will have to change – but how? Paul Gerrard Gerrard Consulting 1 Old Forge Close Maidenhead Berkshire SL6 2RD UK e: w: t: Slide 1Assurance with Intelligence

Paul is the founder and Principal of Gerrard Consulting, a services company focused on increasing the success rate of IT-based projects for clients. He has conducted assignments in all aspects of Software Testing and Quality Assurance. Previously, he has worked as a developer, designer, project manager and consultant for small and large developments using all major technologies and is the webmaster of gerrardconsulting.com and several other websites. Paul has degrees from the Universities of Oxford and London, is Web Secretary for the BCS SIG in Software Testing (SIGIST), Founding Chair of the ISEB Tester Qualification Board and the host/organiser of the UK Test Management Forum conferences. He is a regular speaker at seminars and conferences in the UK, continental Europe and the USA and was recently awarded the “Best Presentation of the Year” prize by the BCS SIGIST. Paul has written many papers and articles, most of which are on the Gerrard website. With Neil Thompson, Paul wrote “Risk-Based E-Business Testing” – the standard text for risk-based testing. Paul Gerrard Assurance with IntelligenceSlide 2

It’s the Benefits, Stupid! Automation frameworks Test Process Improvement is a Waste of Time Software Success Improvement What Makes a Good Tester? Recommendations Agenda Slide 3Assurance with Intelligence

It’s the Benefits, Stupid Slide 4Assurance with Intelligence

Won’t bore you with yet another survey of IT projects that fail - take it for granted, most do But why? We can trace most failures to: - Bad decisions - Decisions that were made too late - Decisions that were not made at all We suggest this happens because stakeholders and project managers lack the right information at the right time We’ll call this information Project Intelligence Why projects fail The frame of reference for making those decisions is often wrong too! Slide 5Assurance with Intelligence

Four-eyed plans Increasingly Inaccurate IT- focused Initial Plans The plan is a model of the project. The real project consists of people, organisation, goals and risks. Slide 6Assurance with Intelligence

The people like to see the broad range of issues covered by the politicians Makes politicians feel important But it’s all froth Team Poster It’s the economy, stupid! benefits Ultimately, our customers are only interested in the benefits of IT Slide 7Assurance with Intelligence

Number of people in IT will drop by 15% by % percent of technology professionals will move to more business-focused roles, concentrating on the use of IT and processes rather than IT delivery In 2010, the typical IT department in a large company will be at least one third smaller than it was in 2000 Departments within business will take on the traditional roles of IT. Gartner predictions Slide 8Assurance with Intelligence

Increasingly, businesses will focus on benefits and will take control of projects or IT completely Projects involving IT will no longer be dominated or managed by IT The disciplines of Benefits Realisation, Goal-Directed Project Management and Project Intelligence will become mainstream Most testers will work for (or come from?) business (Some) Test Managers become PI Managers. Here’s my interpretation… Slide 9Assurance with Intelligence

Automation Frameworks I am very grateful to Susan Windsor for the use of some of her material Slide 10Assurance with Intelligence

Automation frameworks and more complex business requirements Agile development methods mean developers undertake more unit and component testing Growth of outsourced testing to different geographies => greater competition for the roles Testing is in demand, and solutions reduce the number of functional testers required Is the role of the functional tester (who is neither technical nor business specialist) dead? Slide 11Assurance with Intelligence

Focus on technology rather than business needs 80% of functional testing still manual 60% to 70% of automation tools used for non- functional testing Typically, traditional functional automation stops at 100 scripts, regardless of test coverage requirement Critical factors - Cost of implementation and maintenance prohibitive - Insufficient and expensive skills required - Inability to asset share over different technologies Functional test automation is broken! Source: Paul Herzlich, OVUM UK Slide 12Assurance with Intelligence

Home grown frameworks built within organisations to meet business demands Niche suppliers provide frameworks - try Google: - 34,400 exact matches for “test automation framework” - A few are now mature - Latest review by Paul Herzlich (OVUM analyst) Market Leaders such as Mercury developing Business Process Tester (BPT) Business analysts already using them, and use will grow This seems to be the tools industry direction now Slide 13Assurance with Intelligence

Slide 14 Manual Test Execution & Documentation UI Automated Test Execution Tools Test Results Non UI Component Test Execution Harness Requirements Test Management Governance Project ManagementAnalysis & Design Test Case Definition – Automation Framework Test Case Documentation Test Requirements Sign Off Where frameworks fit 14Assurance with Intelligence

Test Process Improvement is a Waste of Time Slide 15Assurance with Intelligence

I want to improve my ( insert any activity here ) _______ people improvement _______ organisation improvement _______ process improvement How to improve…  Changing people (like me) and organisation (like my company) is so hard – let’s not even think about it Slide 16Assurance with Intelligence

There are no “practice” Olympics to determine the best There is no consensus about which practices are best, unless consensus means “people I respect also say they like it” There are practices that are more likely to be considered good and useful than others, within a certain community and assuming a certain context Good practice is not a matter of popularity. It’s a matter of skill and context. The delusion of ‘best practice’ Derived from “No Best Practices”, James Bach, Slide 17Assurance with Intelligence

Google search - “CMM” – 12,100,000 - “CMM Training” – 12,200 - “CMM improves quality” – 4 A recent client… - CMM level 3 and proud of it (chaotic, hero culture) - Hired us to assess their overall s/w process and make recommendations (quality, time to deliver is slipping) recommendations, only 7 adopted – they couldn’t change - How on earth did they get through the CMM 3 audit? The delusion of process models (e.g. CMM) Slide 18Assurance with Intelligence

Slide 19Assurance with Intelligence

People like simple models: - levels of maturity, stepping stones, checklists, roadmaps and outside support for credibility But life is much more complicated, unfortunately “Things should be made as simple as possible, but no simpler” Albert Einstein But process models make improvement simple don’t they? Slide 20Assurance with Intelligence

A big problem with process is it becomes all encompassing Process folk sell process and cast all things in terms of it - They ignore that people who are smart - Smart people succeed regardless of process, not because of it It could be argued, that less smart people need process - (By less smart, we're talking about people who need so much structure and enforced discipline they can only operate in the military, or in prison probably) Is our industry really staffed by such people? Do we really want production-line workers? Do YOU really want to be a production-line worker? People need process? Slide 21Assurance with Intelligence

“I believe that a scientist looking at nonscientific problems is just as dumb as the next guy” “It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong” Richard P. Feynman Physics quotes… Slide 22Assurance with Intelligence

“I believe that a process consultant looking at non- process problems is just as dumb as the next guy” “It doesn't matter how beautiful your process model is, it doesn't matter how smart you are. If it doesn't agree with reality, it's wrong” “Test process improvement is tinkering” Paul Gerrard ;-) You can quote me if you want… Slide 23Assurance with Intelligence

Software Success Improvement Slide 24Assurance with Intelligence

From this… Current Maturity Future Maturity Process Change Current Capability Future Capability Acts of Faith Better Capability = better, faster, cheaper Perceived Results Chain Slide 25Assurance with Intelligence

To this… Current Constraints/ Problems Changed Aspirations Current Capability Future Capability Acts of Change Better Capability = better, faster, cheaper Actual Results Chain Slide 26Assurance with Intelligence

Constraints are fixed headcount, budget, timescales, quality of requirements, contracts etc. Problems are “testing takes too long; too expensive; can’t hire testers; bugs get through” etc. etc.… Aspirations: - Personal: personal development, fulfilment, motivation - Organisational: hero culture to team culture, outsourced, higher consistency, predictability Acts of change are… Constraints, problems and aspirations Slide 27Assurance with Intelligence

Changes in behaviour to address specific problems (time, cost, quality etc.) Targeted personal and team development Infrastructure change (process, techniques, tools, environments) to support the changes Managed Transition… Acts of change – focused on constraints, problems and aspirations Slide 28Assurance with Intelligence

Current behaviour assessed in the context of current problems, goals and constraints Aspirations drive the definition of goals People in the job define and consent to the required changes in behaviour People supported by - Personal/team development plans - Infrastructure investment (process, technology, tools, environment) specific to the change Transitions are managed, not assumed. Principles of change Slide 29Assurance with Intelligence

Establish a sense of urgency Create a guiding coalition Develop a vision and strategy Communicate the change vision Empower broad-based action Generate short term wins Consolidate gains, produce more change Anchor new approaches in the culture. Eight stage change process (Kotter) Derived from “Leading Change”, John Kotter, Slide 30Assurance with Intelligence

Mission Coalition Vision Communication Action Wins Consolidation Anchoring Eight stage change process (after Kotter) Changes identified here This is where your ‘process model’ comes into play Slide 31Assurance with Intelligence

What Makes a Good Tester? Slide 32Assurance with Intelligence

Pedantic Sceptic Nitpicker A perfect tester? Slide 33Assurance with Intelligence

Attributes of a good tester? intelligent persistent systematic pedantic thorough sceptical logical curious articulate conscientious imaginative accurate observant thick-skinned assertive redoubtable stubborn Slide 34Assurance with Intelligence

Attributes of a good tester? intelligent persistent systematic pedantic thorough sceptical logical curious articulate conscientious imaginative accurate observant thick-skinned assertive redoubtable stubborn Personal intelligence and skill Personality to cope with your environment Slide 35Assurance with Intelligence

Intelligence - are they smart enough? Thinking skills - Approach to problem solving; ability to reason Interpersonal skills - Communication, assertiveness, conflict handling, survival, team skills etc. etc. Testing skills - Theory (ISEB/ISTQB Etc.) - Practical (hands on testing) - Technical (technology, business domain etc.) Plus… Personality - Many well-known attributes are usually required. Tester selection criteria Slide 36Assurance with Intelligence

Tester skills (and training budget?) Testing theory (ISEB/ISTQB etc.) - see Thinking skills - verbal reasoning, numerical/abstract reasoning, fault diagnosis, accuracy Testing Practice (Testing Case Studies) - hands-on practical test activity Interpersonal skills - communication, assertiveness, conflict handling, survival, team skills 20% 40% 20% Slide 37Assurance with Intelligence

Summary and Discussion Slide 38Assurance with Intelligence

Take a closer look at benefits realization, results-based management, performance management, project intelligence Become more business-oriented - Be ready to talk their language – benefits and risk - Align your test activities with the need to provide information on benefits and risk to them - Be prepared to be managed by your customer! It’s the benefits, Stupid! Slide 39Assurance with Intelligence

Take a close look at frameworks What is your organisation doing about it? Understand how Frameworks support testing so you have a view, because you will be asked If you are functional tester – evolve or… Look to enhance your skills… - Become an automation/framework specialist - Move towards management - Take on non-functional skills - Get closer to your business/customers Automation frameworks Slide 40Assurance with Intelligence

Don’t take on Test Process Improvement projects - Too many problems in test are caused elsewhere - Testers shouldn’t compensate for other people’s failures - Whole-process is in scope for change - or nothing! Ask why things are the way they are Focus on people, culture and organisation “The art of the possible” – get your suggestions from your practitioners, not a book Follow the 8 step change process (or another transition management method – there are many) Be realistic about benefits, and be prepared to measure them Software Success Improvement Slide 41Assurance with Intelligence

Functional testing skills are a commodity – so move on Have a vision of where you want to be… manager, specialist, consultant, business expert… Following technology changes can be lucrative, but is a never- ending journey Broaden your horizon – thinking, interpersonal, business, management, project management skills Create a development plan – training yes, but look out for practical hands-on, not theory Most learning takes place on the job so choose jobs wisely Find a good coach. Developing, as a tester Slide 42Assurance with Intelligence

gerrardconsulting.com uktmf.com riskbasedtesting.com Good luck in your career! Trends in the Marketplace Testers will have to change – but how? Slide 43Assurance with Intelligence