Day 1 Quality control methods and practice Kees van Berkel Mariëtte Vosmer Jerusalem, July 2013

Response LFS

Non contacts LFS

Refusals LFS

Possible explanations  Decentralized field supervision  Annual supervising cycle:  setting targets in performance agreement  monitoring performance  assessment of performance every six months

Possible explanations (2)  Significant changes in fieldwork strategy  Use of visiting cards  After 3 visits: call to make an appointment  First or second visit on ‘preferred’ time  First visit in first half of fieldwork period  Visit non contacts 6 times  Spread visits across times / days / weeks  Training in gaining cooperation  Monitoring

 Main target: response  Quantitative AND qualitative targets  Target for survey, regions and interviewer  Targets in performance agreements  Assessment of performance 2x p. year  Information monthly or weekly  Distributed to decentralized management Output monitoring

Performance agreement  Interview only face to face  Interview on laptop  Apply required approach strategy  Apply suggestions to gain coöperation  Ask questions literally  Use identification card

Quantitative monitoring  Response, refusals, non contacts  Correct spread visits  Length of the interviews  Prompt return of documents and data  Partial response, panel refusals  Questionnaire indicators, for example:  % people > 65 years  % interviews < 5 minutes  time between interviews < 5 minutes

Quantitative monitoring (2) Based on:  Response and progress data:  Interviewers account for every visit  Datacom every working day  Quality control system:  Information system compiled of all LFS data over a year  Compares scores of interviewers on indicators with the average of region

variabelaverage <<>> average number visits 1,5 1,9 2,3 2,9 3,1 % to panel indeed27,043,372,6 86,7 88,5 % to panel after canvass question39,063,084,7 98,4 98,8 % 65+ in questionnaire 2,5 5,111,4 17,9 19,3 % 65+ in register- - 1,4 3,9 7,2 % proxy 9,720,033,2 43,6 44,8 % partial NR - - 2,9 12,0 23,9 % interv. time < 5 min ,1 6,9 9,4 % 65+ interv.time > 25 min ,2 33,3 40,0 % ratio PQ/interv.time < 60%- - 4,7 11,7 14,1 % time between interv < 5 min ,8 4,2 5,2 % 1e visit in 1e half of the month58,065,585,9 99,1 99,8 % at least 6 visits if no contact 5,344,184,2-- Quantitative control – Standards

Quantitative control – Standards (2) RankingIdIntRegion RespTotal

Qualitative monitoring (RAS)  Did the interviewer visit the household?  Face-to-face with use of laptop?  Correct fieldwork procedure applied?  Comments about interviewer (positive or negative)?  Data correct? (ages, number of household members)  Persuasion to cooperate? Based on: Response Analysis Survey (RAS)

Procedures quality control  Standard procedure:  quarterly global quality control  yearly Random Analysis Survey (RAS)  Standard follow-up:  specific quality control if exceeding standards  second time RAS  Special follow-up:  during 3 months RAS

Procedures quality control (2)  Follow-up:  Decision: sector management  Request: management team or region managers  Commission on follow-up: sector management  Take up with interviewer: region manager in appraisement interview or sector management (grave fault)

Secundairy effects of quality control Improving interviewer performance may lead to problems in analysis Case: Decrease in unemployment figures possibly due to more visits in the evening and weekends.

Conclusion  Set targets for individuals  Monitor and provide information  Describe targets in formal agreements and evaluate  Success is almost guaranteed!