1. Training and accreditation of interviewers to carry out physical measurements on the Millennium Cohort Study Age 11 survey
04/07/2013
Nickie Rose (Ipsos MORI) and Lisa Calderwood (Centre for Longitudinal Studies, Institute of Education)

2. Background
Number of surveys in different countries include physical measurements
One element of a broader range of biomeasures collected alongside survey data
Three main models for biomeasure data collection in social surveys
Medically trained personnel in specially equipped clinics
Home-visit setting with portable equipment
Mobile nurse fieldforce
Specifically trained interviewers
Well-established that field interviewers are able to carry out more common physical measurements
Has advantages if collected alongside other study elements
Higher response rates
Cost effective
But requires additional training and support and appropriate quality control during fieldwork

3. Millennium Cohort Study (MCS)
Cohort study of around 19,000 children born 2000/2001 – home visit- based
Families surveyed when children 9 months, 3 years, 5 years, 7 years and most recently 11 years old
Data collected from main carer (typically mother) and partner (typically father or step-father): interview and self-completion
Direct assessments of children’s cognitive function
From age 7, data collected from cohort children themselves via a self- completion
Data collected from older siblings in some sweeps via self-completion
Some bio-measures collected (e.g. teeth, oral fluid via mouth swabs (not for DNA extraction))
Physical measurements (e.g. height, weight and body fat percentage, waist measurement, physical activity monitoring)

4. This paper
Millennium Cohort Survey (MCS) uses interviewers to collect physical measurements from children
Response rates to the physical measurements are high % at age 7
Challenge to maintaining this at Age 11 survey
11 year olds more self-conscious about being measured and handled by an interviewer
For many Ipsos MORI interviewers it was the first time they would have collected physical measurements
The CLS team were keen to review the approach to training and monitoring interviewers, following best practice on other similar studies
Led to the development of a formal accreditation process and additional early accompaniment for all interviewers
Paper describes how we achieved this in the context of an already very full briefing programme

5. Physical measurements on MCS
At age 11, interviewers required to measure height, weight and body fat
Height and weight used to calculate BMI
Measure of obesity
Body fat provides an estimate of fat-free mass
Children who are overweight tend to become overweight adults
Risks of illness and disease
Psychological consequences
Levels of obesity are increasing
In 1965 National Child Development Study found 11% of 7 year olds were overweight
In 2008 MCS found 21% of 7 years olds were overweight
Major concern for UK Government who are committed to delivering a sustained downward trend in levels of excess weight

6. Experience from other surveys
Accreditation schemes are standard practice in many US-based studies
Health and Retirement Study
National Social Life Health and Aging Project
Fragile Families
National Longitudinal Survey of Youth
More recently adopted in the UK
Health Survey for England
Understanding Society
English Longitudinal Study of Aging
Ensures interviewers are able to accurately follow study protocols

7. What is accreditation?
Accreditation takes place after comprehensive training and practice
Performance is assessed against a set of standardised criteria
Certification checklist
Interviewers pass or fail
Feedback is provided
Interviewers are required to pass before starting work
Benefits
Interviewers are accurate and consistent
Interviewers are confident in abilities
Promotes data quality
Maximises response rates

8. Building on existing MCS training
Existing training for MCS is well-established and robust
3 days in classroom
“Active learning” – taught elements, demonstrations and practice sessions
Practice session with two children of same age as cohort between day 2-3
Home learning before and during briefing
Informal practice administering CAI

9. Briefing the physical measurements
Existing briefing programme
Day 1:
Detailed classroom session on measurement protocols lasting 2 hours 15 minutes
Videos
Demonstration in small groups
Practice measuring height and familiarised themselves with scales operation
Between Day 2-3
Rehearsal with two specifically recruited 11 year olds
Day 3
Feedback on in-home practice sessions
Quiz on measurement protocols

10. Height protocol Height measured using a Leicester stadiometer
Protocol covers
Correctly assembling the height measure
Preparing the child
Correctly positioning child
Taking an accurate measurement
Bring down measuring arm
Move head to achieve Frankfurt Plane
Child to duck and step off
Take reading to nearest completed millimetre
Record in CAI immediately
Dismantle and pack away

11. Height measurement

12. Measuring weight and body fat
Calibrated Tanita scales
Weight
Body fat
Weak electrical current sent around body from one foot to another measures resistance of muscle and fat
Measured for all children except those who cannot stand unaided or were wearing an electronic device such as a pacemaker
If body fat refused able to measure weight only
Check scales measuring in the right mode (kilograms)
Enter child’s age and height (taken from CAI)
Record in CAI immediately

13. Developing the accreditation process
CLS team trained in measuring height by expert at Child Growth Foundation
Worked with a member of Ipsos MORI team to develop protocol and accreditation process
Developed an accreditation checklist based directly on the measurement protocols
Ensured trainers were consistent in their assessment
Tested only on taking an accurate measurement, not tested on other elements e.g. explanations or rapport
Full briefing procedures tested in a session that trained and accredited the briefing team
Height measurement accredited in full
Weight/body fat accreditation involved showing competency with scales

14. Guidelines for interviewer accreditation
General and specific guidelines developed to supplement checklist
General guidelines covered environment and testing conditions
Accreditation should take place in pairs, matched by gender/height
Separate room or quiet area
No feedback, including non-verbal clues may be given during accreditation
Feedback will be given by accreditor to each interviewer separately once both have been observed
Interviewers may use the Physical Measurements Summary Sheet and Frankfurt Plane card
Mistakes could be self-corrected
Subject should remove shoes and be co-operative
Specific guidelines referred to accreditation checklist
Re-sits organised for those not passing

15. Incorporating accreditation into the briefing programme
MCS briefing programme was already very full
Not possible to extend further
Challenge was to include accreditations within existing plan
Interviewers wanted more time to practice
At each briefing up to 20 interviewers to accredit
Considered accrediting during in-home practice sessions but felt inappropriate
Decided accreditation should take place on day 3, following practice
Time and resource constraints led to decision to accredit on height only
Measurement is more complex
Technical issues if not removing socks/tights for weighing
Concerns about weighing peers

16. Incorporating accreditation into the briefing programme
Allocated 40 minutes in schedule for accreditation
In pairs, 15 minutes per interviewer plus feedback
Additional 10 minutes for re-sits
Resources to achieve this were problematic
Overall needed to accredit 309 interviewers over 19 briefing sessions
Up to 8-10 pairs of interviewers per briefing
But only 2 Ipsos MORI briefers available at each briefing
Could take over 3 hours elapsed time
Solution was to accredit supervisors in advance
All attended a quality control session on day 2
Trainers had to be flexible balancing number of interviewers and accreditors and accomodating resits during breaks

17. Fieldwork quality control
Interviewers required support beyond initial practice and training
Interviewer accompaniments
All interviewers were accompanied by a supervisor within first 4 weeks on study
Priority given to those identified at briefing as needing extra support and/or new to this type of research
Modified standard accompaniment form
Supervisors checked adherence to protocols and offered feedback
Exception reporting
Unusual patterns in data by interviewer
Response rates for physical measurements/body fat refusal
Identified interviewers for additional training with equipment /feedback on performance

18. Findings – learning and confidence
This was a successful exercise
Improved training and support for interviewers
Consolidated learning
Confirmed interviewers could follow the protocols
Emphasised importance of following the protocols
All interviewers passed before leaving the briefings
Around four interviewers per briefing were required to resit
Interviewer confidence
Developed confidence in abilities
Reinforced by accompaniments
Interviewers found these invaluable
17 were accompanied twice, 2 were accompanied three times
3 interviewers were removed from the MCS panel as a result

19. Findings – response rates
Response rates to the physical measurements were high
98.4% productive at Age 11 survey
98.8% at Age 7 survey
Just 0.8% refused body fat measurement
A success given expectation that response rates would be more challenging among 11 year olds

20. Findings – practical lessons
Incorporating the accreditations in the limited time available required careful planning
Number of interviewers and accreditors
Provision of adequate space
Briefers needed to be flexible while ensuring approach was standardised
The guidance provided was helpful
Managing interviewers expectations
Ensure interviewers are aware of need and rationale for accreditations in advance
Accreditation of briefers improved quality of training

21. Recommendations
MCS5 team successfully improved training and support for interviewers in collecting physical measurements and ensuring high response rates
Achieved this by
Formalising the skills required
Developing an accreditation process
Training and accrediting the trainers
Providing ongoing support in the field

22. Recommendations Review does suggest some lessons for the future
Allow more time for accreditation and feedback, particularly resits
Could be achieved by running parallel sessions
Slight amendment to accreditation form to indicate pass/fail and ensure that trainers use a separate form for each sitting
Include accreditation of height and weight/body fat
Formalise rules about expiry of accreditation
If working on other studies (which may have different protocols)
Or on same study but for subsequent waves or after an elapsed period

23. Thank you nickie.rose@ipsos.com | l.calderwood@ioe.ac.uk © Ipsos MORI
This work will be carried out in accordance with the requirements of the international quality standard for market research, ISO 20252:2006 and with the Ipsos MORI Terms and Conditions which can be found here