© Copyright 2010 Aqastra1 Dedicated to Testing Excellence Summit 2010 Selecting our Testers and Measuring their Performance Susan Windsor

Slide 2 Discussion Topics Where to start Build a team profile Skills/Competencies Framework Assessment Techniques Education Needs Talent Management

Slide 3 Where to start? You never have a blank “canvas” for staff selection, so what are the challenges? – Existing staff – Existing suppliers – Budget constraints – More.....?

Slide 4 We talk about testers as if they are a “special breed” but it’s the required activities that really count!

Slide 5 And the context is critical too As it always is for your test approach Organisational objectives – Business area, time to market, mission critical Development approaches – Agile, waterfall, and other variants Technical architecture – Level of integration, complexity of estate Test focus areas – Performance, accessibility, integration More…..?

Slide 6 The skills and competencies you need are specific to the context of your project How different are your projects?

Slide 7 Build a Team Profile To meet the needs of a specific project – Use your Project Test Strategy as a start point Ultimately, to meet the needs of your organisation – Use your Organisational Test Strategy as a start point, or combine the profiles from your projects

Slide 8 Lets look at the project first What ever we call the test approach for a project (strategy, framework, plans, etc) and however we decide to document it (or not) - the discussion about “how are we going to test this stuff?” is the important part

Slide 9 Build an Activities Framework Before we think about roles and responsibilities Driven by the specific needs of the project Regardless of the resources available (initially) Appropriate to our stakeholders and organisational objectives

Slide 10 Consider…. Management – planning, reporting, etc Functional – areas of business expertise required Non-functional – performance, integration, etc Test Support – test data, environments, tools, etc Test Methods – structured scripted, exploratory Interpersonal – communication, team working, etc Produce an activities checklist

Slide 11 Skills/Competencies Framework Use this to identify the skills and competencies required to deliver your activities checklist Then map your known resources to the project requirements This will identify your gaps – Can you educate – Can you bring in additional resources – Are you managing the risk of these gaps

Slide 12 Sources for the framework? SFIA – Skills Framework for the Information Age – Used by Government bodies – Very little specific to testing (not always a bad thing!) – 7 levels so a challenge to be pragmatic and apply BCS – Again has 7 levels! – More comprehensive on testing specific activities – Assumes a traditional structured approach – Good place to start Organisational – Most test service organisations should have developed one to manage their staff career development, ask your suppliers if you can use theirs – Take our (next slide) open source framework and build on it for your own organisational needs – it’s open source!

Slide 13 Our skills/competencies framework Is aligned with BCS and SFIA Includes Skills, Competencies and Productivity Factors Has 3 definition levels (familiar, practiced and expert), for a pragmatic approach Includes input from Test Manager community Has been piloted with customers Will continue to evolve (all contributions and feedback welcome) and will always be available to the TMF Community

Slide 14 A handout of the Skills/Competency Framework will be provided to attendees at the Summit during this session

Slide 15 So, we have our Team Profile!

Slide 16 At a glance you identify gaps/issues GapsWhy?

Slide 17 What Next? Revisit the profile, can you compromise? Build specific roles and responsibilities for each individual based on the profile (and communicate this to all involved!) Build “SLA’s” with external entities however formal (or otherwise) you need to be Can any existing resources increase skills? – Or do they already have them and you just don’t know? What additional resources do you need – Write specific role requirements to help identify the individuals

Slide 18 Assessment Techniques When you know nothing about the individual – Resource you’re bringing in, newly assigned from another area, etc. When you are seeking further information about an existing team member to revise your matrix or identify education options

Slide 19 Assessment for New Folk Lets assume you’ve written specific role descriptions covering your needs Potential candidates have provided you with their relevant expertise and experience (rather than just sending you their CV!) And you’ve got a shortlist to review

Slide 20 Prepare for Assessment Aptitude Test to supplement their application Construct set of general questions for all candidates that relate to your priority needs (getting an expert to help you as necessary) so you can compare candidates Identify their potential weakness against your needs and and construct specific questions to help you qualify them

Slide 21 Options to Consider Telephone Interviews – Remember they will have prepared too! – Be sure to have supplementary questions Face to Face Interviews – Are you an expert in the skills you’re seeking? – Get coaching from an expert or have them in the room Practical techniques to verify skills – Known document review – Work alongside a member of your team for an hour doing specific and required tasks – Test known software to identify and document defects – And so on….

Slide 22 Assessment for existing Folk Talk to them! – Most people are candid about their strengths and weaknesses Identify a set of practical tasks to help both of you status of required skills/competencies – Known document reviews – Test known software – Provide them with some test related exercises – Provide them with some communications exercises

Slide 23 When to do all this? Soon as the Test Approach is understood for the project Gives time for any education or resourcing to take place Helps define a workable plan, so has great value

Slide 24 During the Project (and afterwards) Measure their effectiveness against their role and responsibilities – Ask their peers and relevant project stakeholders – Did they need more or less support than planned Do your/their expectations of skills and competencies match reality? – Jointly review their progress Typically people under-estimate strengths – If they are better than planned, make sure you feed this back!

Slide 25 Education Needs Formal training may not be the answer – Not for a specific project, but to meet longer term goals it plays an important part Are there mentors to help support your team – Expertise that can be available on ad-hoc basis Can you run some skills transfer workshops – Bring in some external experts to help you Is there an external network of support available – Such as the TMF community

Slide 26 Impact of Assessment So, you haven’t got everything you need! And, there are no further resources available And there is no budget left for education Revisit your effort estimates based on real people and their abilities – Your overall project plan will change Revisit the risk profile of the project – Clearly communicate the potential project impact

Slide 27 Talent Management Latest term for career development! We looked a building a skills/competency profile for a project, how about building a framework for your organisation next Use this to work with individuals to map our their career goals and objectives Identify education needs and be creative about how to fulfil them

Slide 28 Talent Management Evaluation of progress Build up an asset base of assessment techniques that work for you Learn from feedback when you apply them and continue to improve them – just as you would test assets Include in your staff assessment programme to measure achievement against goals

Slide 29 What else can we do? Lets identify the groups questions and goals for this topic I’ll document and put on the website with this material What follow on information can we build for a future Forum session?