Psychometric Assessment: Tests, test use and preparation 13th July 2017 Loughborough University

Overview Part 1: Part 2: A quick recap on psychometrics Employers’ use of tests Helping students prepare Part 2: New developments Future trends

Psychometric tests “an objective and standardised measure of a sample of behaviour” Anastasi and Urbina (1997) ‘psycho’ = mind ‘metric’ = measurement

Psychometric tests administered in a standardised way scored and interpreted in a standardised way reliable – meaning they are accurate valid – meaning they are appropriate for purpose rigorously developed according to best practice standards

Psychometric tests

Types of tests & assessment Cognitive tests – learning, reasoning, capacity (what) Personality – style, preferences (how) Motivation – drive, energy, engagement (why) SJTs / simulations – judgement, practical knowledge, ‘common sense’ (what & how)

Cognitive tests Focus of much of today as they are the most… widely used widely researched open to preparation feared! They cover tests of ability, aptitude and attainment

Types of cognitive test Ability tests Assess what a person is capable of doing Typically involve ‘reasoning’ or problem-solving May assess general or specific abilities Abstract ability Verbal ability ‘g’ or general ability Numerical ability

Types of cognitive test Attainment tests Assess what a person has learnt e.g. educational qualifications, accountancy exams Influenced by ability, so association Aptitude tests Abilities related to acquiring specific skills Future-oriented, i.e. used to predict In practice, often similar to ability tests

Personality and motivation Self-report measures No(?) ‘right’ or ‘wrong’ answers Assess typical behaviours Personality - distinction between ‘trait’ and ‘type’ Motivation related to drives and values – increasing emphasis, especially on fit

SJTs Can assess behaviours, skills, competencies Based on understanding and responding to scenarios Provide ‘low fidelity simulations’ Relatively distinct from other assessments

SJTs

Why are tests used? Objectivity Accuracy Validity ‘Level playing field’ Structured interviews Astrology Unstructured interviews Graphology SJTs Ability tests Self-assessment 0 Effectiveness 1 References Personality assessments Assessment centres

How good is good enough? Graduate population Employer’s local norm General population Graduate population Employer’s local norm Test score

Popularity CIPD Resourcing and Talent (2017) Competency-based interviews – 78% Interviews on contents of CV / app form 74% General ability tests – 41% Assessment centres – 39% Personality / aptitude questionnaires – 35% Online tests – 23% Gamification – 2% Based on 1068 UK organisations

Popularity Over 95% of FTSE100 companies c70% of SMEs Tendency for greater use in private sector than public or voluntary sectors

Popularity – Gartner (formerly SHL) Assessed more than 5 million people in over 160 countries in the last 12 months Worked with 10,000 customers More than 60% of the Fortune Global 500 used SHL: 80% of Technology companies 80% of Food and Beverage companies 70% of Financial Services companies More than 80% of the FTSE 100 used SHL, including every Telecommunications company Current estimate of 25 – 30 million tests per year

Preparation and test performance or Practice ≠ Preparation

Research on test preparation Almost exclusively on cognitive tests and ‘exams’ Partial and incomplete Assumptions and ambiguities

Purpose of testing - employers Efficiency Managing risk Creating a ‘level playing field’ Fairness Understanding potential Finding the right fit Attraction

Purpose of testing – job seekers To ‘show myself off’ – it’s an opportunity Do I have the right knowledge / skills / abilities? Will I fit in? Do I want this job enough?

Purpose of testing It’s a two-way thing… …though loaded in favour of the employer It helps if it is seen as an opportunity, not a threat

What the research tells us - ability ‘Familiarisation’ vs ‘coaching’ Re-test effect, c0.5 standard deviations Research during the 1980s by ETS Test-specific familiarisation – modest effect Coaching – variable, not as big as claimed Numerical more susceptible than verbal Effect of test complexity Well designed prep, c4 hours, as effective as coaching courses

What the research tells us – personality & motivation Less coachable than ability – job specific Natural changes under assessment conditions ‘Gaming’ the assessment Fit Validation through interview

What the research tells us - SJTs Very little good research available Re-test effects similar to cognitive tests Familiarity is likely to be important – variation in question formats Know the company – what do they want? Typical behaviour and fit

Helping students prepare Action Exploration Reflection Creation

Action What do I know about tests? What tests am I being asked to take? Get some practice

Reflection Key questions to ask: How did I do? Did I understand the instructions properly? How did I feel – confidence vs anxiety? Did I manage speed vs accurately well? What skills do I need to develop?

Reflection Optimum performance Accuracy Speed

Creation Use Reflection points to set targets: Know the test and its requirements Confident and positive Develop speed and accuracy Brush-up on knowledge / skills assessed by test Identify gaps in current knowledge

Exploration Put it into practice Practice tests Information Use feedback to inform approach

Action Get some practice… … and apply what I now know / can do

Test-taking skills Test performance Test-taking skills Ability Low High Test-taking skills Weak Strong Ability

Any questions?

Trends in Psychometric Assessment 13th July 2017 Loughborough University

Overview Developments with current tests Strengths-based assessment Game-based assessment Looking further ahead

Developments with current tests More tests at all stages of recruitment Technology is everywhere, increasing accessibility Testing earlier on in recruitment process Ability tests High use Online, unsupervised Validation? Item banks and adaptive testing

Developments with current tests Personality assessment Increasingly used as a screening tool Bespoke versions, focus on key attributes Can be used to derive competencies SJTs Typically used as an early filter Most likely to be bespoke to organisation Vary in terms of ‘fidelity’ Most advanced ones are ‘game-like’

Going mobile 45% of job seekers use their mobile to search for jobs daily 89% believe a mobile device is an important tool for job searching 48% think mobile will be the most common way to search for jobs in two years or less 90% of the Fortune 500 company career sites do not support a mobile applications (Glassdoor surveys, 2014)

Tests going mobile

Strengths-based assessment Identifies individual strengths, interests, motivators May be an assessment or interview Part of a wider recruitment process Predictive of fit and retention Open to preparation?

Strengths-based assessment

Game-based assessment Some definitions Games Gamification Game-based assessments Attractive Equitable? Valid?

Looking further ahead Diversity vs validity Diversity is increasingly important to organisations Traditional tests, esp ability, are not great here Personality, simulations, GBAs are better here… … but are they equally valid? AI / natural language processing Digital footprint analysis Up to 90% of employers review candidates’ social media profiles Rise of companies offering social media reports

Looking further ahead

Any questions? www.realisepotential.com www.bpstesttraining.co.uk