MICS4 Survey Design Workshop Multiple Indicator Cluster Surveys Survey Design Workshop The MICS3 Evaluation

MICS4 Survey Design Workshop MICS3 Evaluation Commissioned by UNICEF and carried out by John Snow Inc. during

MICS4 Survey Design Workshop Methodology used Global online survey (350 respondents) Key informant interviews Document review In-depth study in 8 countries Data quality assessment

MICS4 Survey Design Workshop Findings MICS is seen as a key data source Data quality at acceptable levels, on par with other survey programmes (e.g. DHS), in most countries examined Tools (manuals, guidelines) technical assistance (workshops, online etc.) of high standard, and highly valued

MICS4 Survey Design Workshop Findings Strong evidence of data use at global level Progress in data use at country level, but the link to programme action and policy development not an evident pattern Wide-spread perception that MICS has helped to build local capacity

MICS4 Survey Design Workshop Findings Final reports are often delayed for long periods, to the point that the data are no longer seen as timely Significant quality lapses and deviations from accepted norms in sampling and fieldwork procedures found in several countries Quality problems are a result of overlooking basic standards of survey implementation

MICS4 Survey Design Workshop From Evaluation Report: The MICS3 manual and regional workshops give specific guidelines regarding how to conduct the field work and how to maintain quality control. The evaluation found significant deviation from these practices in the countries studied, which occurred despite the fact that most countries received technical assistance from UNICEF and rated it highly…

MICS4 Survey Design Workshop From Evaluation Report: When countries adhered to the MICS3 data processing protocols and made use of the support provided, data processing appears to have gone well.

MICS4 Survey Design Workshop Overlooking basic standards Deviations from MICS3 recommended practices (examples): –Out-of-date or incomplete household listings –Shortened training in many cases –Ad hoc translation in the field in some cases; no pre-test –Team composition/supervision/use of editors

MICS4 Survey Design Workshop Contributing to delays and quality issues Over-loaded questionnaires, non-MICS additions, inappropriate use of modules, lack of understanding Lack of capacity in sampling, data processing, report writing Problems discovered (too) late

MICS4 Survey Design Workshop From Evaluation Report: In some instances, the need for skilled technical advice is only identified after key decisions have been taken and implemented necessitating rescue operations of sorts to salvage the effort.

MICS4 Survey Design Workshop Some conclusions To improve data quality; standards must be adhered to. This can only happen by providing: Timely and high quality in-country support Regional level coordination Good communication Realistic budgets to avoid short-cuts

MICS4 Survey Design Workshop Some conclusions To improve data use; Improve quality Improve timeliness Set realistic expectations – use interim products (preliminary findings report)

MICS4 Survey Design Workshop MICS4 Strengthened technical assistance system, with increased in-country support Strengthened coordination at regional level Increased funding

MICS4 Survey Design Workshop MICS4 Continue good practices – manual, workshops Easier-to-use and customize survey tools