1. Day 1 Quality control methods and practice Kees van Berkel Mariëtte Vosmer Jerusalem, 21-24 July 2013

2. Response LFS 1973-2010

3. Non contacts LFS 1999-2010

4. Refusals LFS 1999-2010

5. Possible explanations  Decentralized field supervision  Annual supervising cycle:  setting targets in performance agreement  monitoring performance  assessment of performance every six months

6. Possible explanations (2)  Significant changes in fieldwork strategy  Use of visiting cards  After 3 visits: call to make an appointment  First or second visit on ‘preferred’ time  First visit in first half of fieldwork period  Visit non contacts 6 times  Spread visits across times / days / weeks  Training in gaining cooperation  Monitoring

7.  Main target: response  Quantitative AND qualitative targets  Target for survey, regions and interviewer  Targets in performance agreements  Assessment of performance 2x p. year  Information monthly or weekly  Distributed to decentralized management Output monitoring

8. Performance agreement  Interview only face to face  Interview on laptop  Apply required approach strategy  Apply suggestions to gain coöperation  Ask questions literally  Use identification card

9. Quantitative monitoring  Response, refusals, non contacts  Correct spread visits  Length of the interviews  Prompt return of documents and data  Partial response, panel refusals  Questionnaire indicators, for example:  % people > 65 years  % interviews < 5 minutes  time between interviews < 5 minutes

10. Quantitative monitoring (2) Based on:  Response and progress data:  Interviewers account for every visit  Datacom every working day  Quality control system:  Information system compiled of all LFS data over a year  Compares scores of interviewers on indicators with the average of region

11. variabelaverage <<>> average number visits 1,5 1,9 2,3 2,9 3,1 % to panel indeed27,043,372,6 86,7 88,5 % to panel after canvass question39,063,084,7 98,4 98,8 % 65+ in questionnaire 2,5 5,111,4 17,9 19,3 % 65+ in register- - 1,4 3,9 7,2 % proxy 9,720,033,2 43,6 44,8 % partial NR - - 2,9 12,0 23,9 % interv. time < 5 min. - - 2,1 6,9 9,4 % 65+ interv.time > 25 min. - - 8,2 33,3 40,0 % ratio PQ/interv.time < 60%- - 4,7 11,7 14,1 % time between interv < 5 min. - - 0,8 4,2 5,2 % 1e visit in 1e half of the month58,065,585,9 99,1 99,8 % at least 6 visits if no contact 5,344,184,2-- Quantitative control – Standards

12. Quantitative control – Standards (2) RankingIdIntRegion12345678910111213RespTotal 188237 9 219 289176 5 8118 389177 1 817 488047 5 94 8 588527 5 249 8 688148 2 189 7 788373 3 132 5 888645 1 155 5 988662 6 119 5 1089092 1 32 4 1189111 6 103 4 1288026 1 175 3 1388499 2 137 3 1488517 3 191 3 1588518 5 225 3

13. Qualitative monitoring (RAS)  Did the interviewer visit the household?  Face-to-face with use of laptop?  Correct fieldwork procedure applied?  Comments about interviewer (positive or negative)?  Data correct? (ages, number of household members)  Persuasion to cooperate? Based on: Response Analysis Survey (RAS)

14. Procedures quality control  Standard procedure:  quarterly global quality control  yearly Random Analysis Survey (RAS)  Standard follow-up:  specific quality control if exceeding standards  second time RAS  Special follow-up:  during 3 months RAS

15. Procedures quality control (2)  Follow-up:  Decision: sector management  Request: management team or region managers  Commission on follow-up: sector management  Take up with interviewer: region manager in appraisement interview or sector management (grave fault)

16. Secundairy effects of quality control Improving interviewer performance may lead to problems in analysis Case: Decrease in unemployment figures possibly due to more visits in the evening and weekends.

17. Conclusion  Set targets for individuals  Monitor and provide information  Describe targets in formal agreements and evaluate  Success is almost guaranteed!