1. Psychometric Assessment: Tests, test use and preparation
13th July 2017
Loughborough University

2. Overview Part 1: Part 2: A quick recap on psychometrics
Employers’ use of tests
Helping students prepare
Part 2:
New developments
Future trends

3. Psychometric tests
“an objective and standardised measure of a sample of behaviour”
Anastasi and Urbina (1997)
‘psycho’ = mind
‘metric’ = measurement

4. Psychometric tests administered in a standardised way
scored and interpreted in a standardised way
reliable – meaning they are accurate
valid – meaning they are appropriate for purpose
rigorously developed according to best practice standards

5. Psychometric tests

6. Types of tests & assessment
Cognitive tests – learning, reasoning, capacity (what)
Personality – style, preferences (how)
Motivation – drive, energy, engagement (why)
SJTs / simulations – judgement, practical knowledge, ‘common sense’ (what & how)

7. Cognitive tests Focus of much of today as they are the most…
widely used
widely researched
open to preparation
feared!
They cover tests of ability, aptitude and attainment

8. Types of cognitive test
Ability tests
Assess what a person is capable of doing
Typically involve ‘reasoning’ or problem-solving
May assess general or specific abilities
Abstract ability
Verbal ability
‘g’ or general ability
Numerical ability

9. Types of cognitive test
Attainment tests
Assess what a person has learnt
e.g. educational qualifications, accountancy exams
Influenced by ability, so association
Aptitude tests
Abilities related to acquiring specific skills
Future-oriented, i.e. used to predict
In practice, often similar to ability tests

10. Personality and motivation
Self-report measures
No(?) ‘right’ or ‘wrong’ answers
Assess typical behaviours
Personality - distinction between ‘trait’ and ‘type’
Motivation related to drives and values – increasing emphasis, especially on fit

11. SJTs Can assess behaviours, skills, competencies
Based on understanding and responding to scenarios
Provide ‘low fidelity simulations’
Relatively distinct from other assessments

12. SJTs

13. Why are tests used? Objectivity Accuracy Validity
‘Level playing field’
Structured interviews
Astrology
Unstructured interviews
Graphology
SJTs
Ability tests
Self-assessment
0 Effectiveness
References
Personality assessments
Assessment centres

14. How good is good enough? Graduate population Employer’s local norm
General population
Graduate population
Employer’s local norm
Test score

15. Popularity CIPD Resourcing and Talent (2017)
Competency-based interviews – 78%
Interviews on contents of CV / app form 74%
General ability tests – 41%
Assessment centres – 39%
Personality / aptitude questionnaires – 35%
Online tests – 23%
Gamification – 2%
Based on 1068 UK organisations

16. Popularity Over 95% of FTSE100 companies c70% of SMEs
Tendency for greater use in private sector than public or voluntary sectors

17. Popularity – Gartner (formerly SHL)
Assessed more than 5 million people in over 160 countries in the last 12 months
Worked with 10,000 customers
More than 60% of the Fortune Global 500 used SHL:
80% of Technology companies
80% of Food and Beverage companies
70% of Financial Services companies
More than 80% of the FTSE 100 used SHL, including every Telecommunications company
Current estimate of 25 – 30 million tests per year

18. Preparation and test performance or Practice ≠ Preparation

19. Research on test preparation
Almost exclusively on cognitive tests and ‘exams’
Partial and incomplete
Assumptions and ambiguities

20. Purpose of testing - employers
Efficiency
Managing risk
Creating a ‘level playing field’
Fairness
Understanding potential
Finding the right fit
Attraction

21. Purpose of testing – job seekers
To ‘show myself off’ – it’s an opportunity
Do I have the right knowledge / skills / abilities?
Will I fit in?
Do I want this job enough?

22. Purpose of testing It’s a two-way thing…
…though loaded in favour of the employer
It helps if it is seen as an opportunity, not a threat

23. What the research tells us - ability
‘Familiarisation’ vs ‘coaching’
Re-test effect, c0.5 standard deviations
Research during the 1980s by ETS
Test-specific familiarisation – modest effect
Coaching – variable, not as big as claimed
Numerical more susceptible than verbal
Effect of test complexity
Well designed prep, c4 hours, as effective as coaching courses

24. What the research tells us – personality & motivation
Less coachable than ability – job specific
Natural changes under assessment conditions
‘Gaming’ the assessment
Fit
Validation through interview

25. What the research tells us - SJTs
Very little good research available
Re-test effects similar to cognitive tests
Familiarity is likely to be important – variation in question formats
Know the company – what do they want?
Typical behaviour and fit

26. Helping students prepare
Action
Exploration
Reflection
Creation

27. Action What do I know about tests?
What tests am I being asked to take?
Get some practice

28. Reflection Key questions to ask: How did I do?
Did I understand the instructions properly?
How did I feel – confidence vs anxiety?
Did I manage speed vs accurately well?
What skills do I need to develop?

29. Reflection
Optimum performance
Accuracy
Speed

30. Creation Use Reflection points to set targets:
Know the test and its requirements
Confident and positive
Develop speed and accuracy
Brush-up on knowledge / skills assessed by test
Identify gaps in current knowledge

31. Exploration Put it into practice Practice tests Information
Use feedback to inform approach

32. Action
Get some practice…
… and apply what I now know / can do

33. Test-taking skills Test performance Test-taking skills Ability
Low High
Test-taking skills
Weak Strong
Ability

34. Any questions?

35. Trends in Psychometric Assessment
13th July 2017
Loughborough University

36. Overview Developments with current tests Strengths-based assessment
Game-based assessment
Looking further ahead

37. Developments with current tests
More tests at all stages of recruitment
Technology is everywhere, increasing accessibility
Testing earlier on in recruitment process
Ability tests
High use
Online, unsupervised
Validation?
Item banks and adaptive testing

38. Developments with current tests
Personality assessment
Increasingly used as a screening tool
Bespoke versions, focus on key attributes
Can be used to derive competencies
SJTs
Typically used as an early filter
Most likely to be bespoke to organisation
Vary in terms of ‘fidelity’
Most advanced ones are ‘game-like’

39. Going mobile
45% of job seekers use their mobile to search for jobs daily
89% believe a mobile device is an important tool for job searching
48% think mobile will be the most common way to search for jobs in two years or less
90% of the Fortune 500 company career sites do not support a mobile applications
(Glassdoor surveys, 2014)

40. Tests going mobile

41. Strengths-based assessment
Identifies individual strengths, interests, motivators
May be an assessment or interview
Part of a wider recruitment process
Predictive of fit and retention
Open to preparation?

42. Strengths-based assessment

43. Game-based assessment
Some definitions
Games
Gamification
Game-based assessments
Attractive
Equitable?
Valid?

44. Looking further ahead Diversity vs validity
Diversity is increasingly important to organisations
Traditional tests, esp ability, are not great here
Personality, simulations, GBAs are better here…
… but are they equally valid?
AI / natural language processing
Digital footprint analysis
Up to 90% of employers review candidates’ social media profiles
Rise of companies offering social media reports

45. Looking further ahead

46. Any questions?