Data sources and collection methods Ken Mease Cairo, June 2009

What types of Data? A thorough assessment may well include: Archival and secondary data Survey data Quantitative and qualitative approaches and data It also will likely include de jure and de facto information

Qualitative and Quantitative data There are basically two types of data: qualitative and quantitative Qualitative data are usually text or words and quantitative data - numbers Qualitative approaches, if conducted in a rigorous manner, require more skill than most quantitative approaches

Qualitative Data and Approaches Qualitative approaches provide text data, but increasingly audio, video and images They are more time consuming to analyze Text management software, such as NVIVO, AtlasTI and AnSWR (available free at Coding is a very subjective process and open to various problems, such as investigator bias or a lack of inter-coder reliability

Quantitative Data and Approaches Quantitative approaches provide increased rigor by investigating relationships at known levels of probability They are easier to analyze because researchers use standard, replicable techniques Common software includes SPSS, Stata and SAS

Sources and Types of Governance Data Country reports/desk studies Cross-country (region)comparative surveys Expert assessments Government data Household surveys Mass opinion surveys Key Stakeholder Surveys Media Indicators Business surveys

Combining different sources and methods Combining methodologies and types of data often provides the most useful results. Combining archival information and administrative data with original qualitative and quantitative survey data allows for triangulation This approach can increase the level of professionalism, credibility and legitimacy.

Administrative, archival and secondary These data, both qualitative and quantitative, objective, reported events, perception and proxies come from a variety of sources: Narrative reports, administrative data and other information routinely collected by government ministries The constitution, laws, policies and legislation Statistics and data gathered by NGOs, international organizations and academics.

Data Mapping It is the best way to identify existing data - accessibility, quality and gaps One tool available to assist with this process is the IMF’s Data Quality Assessment Framework A senior academic conducted the data mapping exercise in Zambia and was very valuable

Survey Data Qualitative and Quantitative Sample sizes can range from 20 to 20,000 or more Costs can range accordingly There are a range of options for who collects the data – independent surveys firms, academics, NSOs Perception and reported events data

Different Types of Interviewing Structured interviews use an identical instrument for each respondent, Interviewers are trained and have explicit instructions. This technique uses primarily structured questions with fixed response sets and very few open-ended questions

Types of Interviewing Semi-structured interviews use a written list of questions that need to be covered in a particular order The questions are often developed from informal discussions and focus groups. They can include open-ended and/or more structured questions Ideal when working with elites, managers, bureaucrats and other people who have limited time

Types of surveys and data collection methods Face-to-face data collection is likely the best option in most developing countries It is also the most expensive and time- consuming It requires professional management of trained interviewers, sampling and other aspects of the study

Mail surveys Work well only if the postal system is reliable The questionnaire must be carefully designed for self-administration, and there should not be too many language issues The cost is usually quite reasonable It misses the homeless and other vulnerable groups who may not have a valid mailing address

Internet surveys Require special planning and design Very good for certain populations These surveys can suffer from low response rates. Developing and implementing internet surveys has become very reasonable in terms of cost Very reasonably priced internet services are available - Survey Gizmo at

Telephone surveys May have coverage issues in most countries. In many countries, people do not have a phone in their home, and if they do have a phone, it is usually a mobile or cell phone Most cell phone providers do not make telephone numbers available for use in random-digit-dial surveys.

Focus groups Can generate information about the background conditions surrounding governance issues. Focus groups are usually efficient in terms of time and money. They are highly participatory and have the potential of generating solutions to the problems identified by the group members It is very demanding and requires highly skilled coordinators.

Focus groups Make individual ratings insignificant Accuracy can suffer, as some individuals may not feel comfortable to speak up in public, while others are hard to keep quiet Focus groups yield less systematic results Focus groups are best used to identify issues and develop surveys rather than as the only source of data

Who collects the data Outside contractors – surveys and desk studies Local survey researchers Local academics for desk studies National Statistical Offices Government ministries NGOs/CSOs

Data collection and dissemination exercise Pick a Governance issue in your country Choose the levels and types of the data you will collect – Macro, Micro, De jure and De facto Identify the types and sources of data Choose the data collection method/s Identify who will collect the data Select several options for sharing the results with different audiences