Slide 1 Intelligent Testing, Improvement and Assurance Susan Windsor Principal Gerrard Consulting Limited +44 (0) Web: gerrardconsulting.com Test Assurance Coming of Age?

Why are we here? When I first started talking about Test Assurance a few years ago, there were lots of blank looks but it’s become a hot topic now. Economic climate? I’ll share some of my experiences with you to illustrate potential benefits and pitfalls. Perhaps you’re interested in this as a career option, or maybe it could help your project or organisation? Slide 2 Intelligent Testing, Improvement and Assurance

Topics What is Test Assurance? Approach to Analysis Delivery Options Stakeholder Benefits Building your Development Plan Meeting your objectives Slide 3 Intelligent Testing, Improvement and Assurance

What is Test Assurance? Definition (1) Three main styles – Internal function with multiple project scope – External function to provide independent assurance on thoroughness, completeness and quality of testing – External function to provide guidance on improved effectiveness (e.g. project in difficulty) Slide 4 Intelligent Testing, Improvement and Assurance

What is Test Assurance? Definition (2) Key characteristics – Insurance policy; if everything always went well it wouldn’t be needed – Never includes delivery responsibility – Always relates to implementation – Demands direct route to senior stakeholders Slide 5 Intelligent Testing, Improvement and Assurance

What is Test Assurance? Definition (3) Terms of reference for recent assignment – Troubled project – Can send you a copy Example Job Description – Use now to explore scope – Use to identify career development path – Need to be credible and make a difference quickly to be accepted – Can send you a copy Slide 6 Intelligent Testing, Improvement and Assurance

What is Test Assurance? Activities (1 and 2 of 6) Establishing the requirement – Meet your stakeholder and understand their goals for you – Start of project/programme or at any point for a project in trouble Analysis – How will you deliver your goals? – What is the current status? – What are the typical hot spots? Gaps, duplication, environments, contracts, staffing, etc. – How long have you got to demonstrate your value? Could be an hour, a day or a week – won’t be longer – We’ll explore a model for analysis shortly Slide 7 Intelligent Testing, Improvement and Assurance

What is Test Assurance? Activities (3 and 4 of 6) Building your Team – Sometimes you have to do this alone – Is there administration support available? – Do you need specialists to support you? If so ask for them immediately – Write a short terms of reference to include your requirements, plans and deliverables - get your stakeholder to sign it and circulate to everyone on the project Understanding your impact – Most people will not understand why you are there – Everyone you need to talk to is busy and may also be under extreme stress – Some people will not tell you the whole truth and may mislead you – There is always organisational politics – Human nature to believe “we’ll catch up”, “lets fix this in-house” Slide 8 Intelligent Testing, Improvement and Assurance

What is Test Assurance? Activities (5 and 6 of 6) Building relationships – Have a clear mandate that you can talk to anyone you want to that is related to the project/programme and senior people in the organisation – Ensure everyone knows who your sponsor is – Accept you may not be popular but you need co-operation to be successful Closure – You will be asked for, and expected to give, your opinion on The overall testing that’s been undertaken Which issues remain outstanding and require action How to make a better start next time Slide 9 Intelligent Testing, Improvement and Assurance

Approach to Analysis Consulting Model – An assessment method designed to support formal reviews of testing effectiveness – But now has to be second nature, rapid analysis essential – It takes a long time to write a short letter! Information Gathering – As a minimum read project start-up documentation; test strategy; test plans, etc. – Look for “out of scope” and “assumptions”. Supplier Review – What is their brief; how are they managed; what are their objectives – Use a supplier risk assessment checklist to seek out priority areas Typical Project Issues – Gaps and/or duplications in test activities – Environments; Test data – Skill levels; Staffing numbers – Levels of reporting – Role of Project Office Slide 10 Intelligent Testing, Improvement and Assurance

Basic Consulting Model Slide 11 Intelligence Testing, Improvement and Assurance Identify Actions Plan the transition Implement the change Baseline Current Assignment Culture Methods Technology Skills Organization Measurement Improve "How do we get there ?" "Do it !" "Where are we now ?" Where do we want to go next?

Expand dimensions to suit your task Slide 12 Intelligence Testing, Improvement and Assurance How effective is the project organisation; are there any communications issues; is the meeting culture appropriate; are there activities not owned by anyone; does everyone understand their role Is the technology appropriate; is it implemented and supported; what issues exist on the ground using it; Assignment Culture Methods Technology Skills Organisation Measurement Are there too many/few staff assigned; do they have the skills/competencies required; is the level of management support sufficient Are the controls and measurements sufficient and appropriate; is the timing appropriate; how does information change as it migrates; is it accurate Does everyone on the team have a common understanding and commitment to the project goals What could inhibit these being achieved Is everyone using the same methods; do they understand the methods; are they appropriate for this project

Identify Realistic Recommendations This is a Must Have! Relate findings to the context of the project; include high level architecture diagrams and organisation charts and mark the “hot-spots” Group findings into a small number of areas or “hot-spots” Rank the findings in each area by priority; consider impact on key project goals (always include timescale); don’t forget financial implications For the top “3” in each area give specific solutions; ask for exactly what you need; give options where possible As a minimum you must be prepared to verify that the reporting and status provided is accurate Slide 13 Intelligent Testing, Improvement and Assurance Remember your time line – 1 hour/day/week!

Delivery Options External Consultant or Internal Assurance Single project/programme or all current projects/programmes Initial baseline and/or review Recommended actions Ongoing assurance Slide 14 Intelligent Testing, Improvement and Assurance

Stakeholder Benefits Gains a project independent and balanced view of the status and quality of testing Builds confidence in the predictability of implementation Provides rapid, accurate and valuable information to base their decisions upon Provides a fast path to key issues for them to resolve Slide 15 Intelligent Testing, Improvement and Assurance

Building your Development Plan? Bit of homework! How could you develop your skills and competencies; can you identify specific ones to work on; where can you find support for this How can you get relevant experience; could you use the consulting model to review existing projects; can you brief senior managers on the value of the function; can you get in a specialist to work alongside and achieve skills transfer Finding a mentor or an expert; who do you know that can already do this; how can you learn from them Slide 16 Intelligent Testing, Improvement and Assurance

And Finally……. Ask me for any support materials you’d like – Job Description – Terms of Reference – Supplier Risk Assessment – Examples of suitable reporting formats Feel free to ask me any questions Contact me on Slide 17 Intelligent Testing, Improvement and Assurance