Metrics for Marketing Data Collection Methodologies, Systems, and Considerations There is no one methodology or system that can work for everyone, but all systems can be improved. This presentation will introduce you to the processes, but the real work is going to be through brainstorming, applying these principles, and making an effort to expand and improve your data collection systems.

Goals for this Session To approach data collection strategically and systematically Identify appropriate, practical methods for diverse data collection needs Understand and analyze the costs associated with data collection Commit to next steps in data collection

Conceptualizing the Data Collection Process Identify business goals Identify data gaps Select/design appropriate methodology Select/design appropriate data management system Collect, Clean, Validate, and Analyze data Make effective decisions In reality, the data collection process starts long before you actually collect the data. A good system is conceptualized strategically from the beginning with properly defining your business goals and working backwards to determine what information you need to inform those goals. The end goal is all about making effective decisions

Considerations in Data Collection Data Source Type of Data Cost Staff Resources Quality/Accuracy Time Frequency Logistics Extensiveness of data needs Please keep these in the back of your mind as we go through each of these components Data source: where do you need to source the data from? Primary vs. Secondary; Farmers; Input Suppiers; Traders; Offtakers; Others? Type of information: Quantitative vs. Qualitative? Cost: how important is this information to your business model and what are you willing to pay for it? ROI? Equipment, Staff, Logistics (Transportation) Staff Resources: do you have staff available for data collection and analysis? Or do you need to hire externally? Quality/Accuracy: what margin of error are you willing to accept? Time: how quickly do you need this information? Frequency: how often do you need to collect this data to make informed decisions? Ongoing? Or periodic/interval? Logistics: what means of communication is required to get this data? does it require a face-to-face interview? Transport? Extensiveness of data needs: how many questions do you need to answer and in what depth? Internet Access: do you need offline capability? Can the process be automated?

Identifying Data Sources Secondary vs. Primary Secondary Sources Government agencies (Ministries of Agriculture, Trade; Bureaus of Statistics) Donors, donor-funded projects (ZOI Reports) Online databases, existing studies Primary Sources Farmers Agrodealers Traders Buyers/off-takers Often best to start with Secondary Sources as they are free; understand what already exists and then use primary data Secondary sources are often flawed, however, so generating primary data independently is very important

Selecting Appropriate Data Type Quantitative vs. Qualitative Quantitative Objective, quantifiable measurements Semi-structured interviews with close-ended questions; self-administered through email, telephone Qualitative Understanding context, client behavior Unstructured, in-depth interviews with open-ended questions; direct observation; focus groups Mixed Methods Ultimately the best and most comprehensive methodologies combine aspects of both (mixed methods). What good is quantitative data unless we fully understand the contextual clues surrounding the data? And what good is qualitative data unless we can pair it with something tangible and measurable for target setting Qualitative: In some cases qualitative data can be more valuable; lower cost; smaller sample sizes

Data Collection Methodologies Census vs. Sampling Census: Collect data on all subjects (i.e. clients, beneficiaries) Appropriate for small populations (<250) Appropriate for information that can be quickly/easily ascertained (i.e. Name, ID, Gender, Age, Telephone, Location) High quality, high cost Sampling: Collect data on a representative sub-group of your population Appropriate for large populations High quality, low to medium cost Census: Certain information is recommended to have on a census basis: i.e. primarily client profiles (Name, ID, Telephone, Location) Census is obviously more time consuming, so it needs to be accompanied by automated tools There are generally trade-offs between cost and data quality, but not always Sampling is the collection of data through a representative sub-group of beneficiaries or clients, selected at random

Sampling Methods Random Systematic Purposive Gold standard; high level of representation; highest accuracy of results Logistically more challenging, resource intensive Example: Off-taker randomly selects smallholder suppliers to sample plots and estimate harvest volumes Systematic A form of random, probability sampling Can be done without lists, but subject to bias Example: Input supplier interviews every 10th customer entering a store to determine reasons for/not purchasing improved seed Purposive A form of non-probability sampling Not representative; results cannot be generalized Subject to bias, but cost-effective Example: A financial service provider interviews select youth clients to understand specifically the barriers to youth accessing finance

Example of Simple Random Sampling Total Population Sample Population

Sampling: Basic Principles The goal of sampling is representation achieved through randomization; to generalize results to a broader population. Accuracy of data measured through confidence level/margin of error. For populations greater than 20,000, a sample size between 350-400 is generally sufficient. Sample sizes vary for quantitative vs. qualitative data collection. Population Confidence Level (95%)/ Margin of Error (5%) Confidence Level (90%)/ Margin of Error (10%) 500 218 60 1,000 278 64 5,000 357 67 10,000 370 68 25,000 379 For accurate estimates, randomization is key Key to sampling is for it to be representative Higher degree of confidence / lower margin of error = larger sample size Populations more than 20,000 and above yield the same sample sizes There are a lot of considerations when determining sample sizes (range of potential responses, homogeneity of the respondents) Estimates listed above are for calculations of percentages or proportions If it is determined that sampling is the right approach, rather than getting lost in these calculations, it is best to consult professionals Another approach is to figure out how many you can potentially survey with your given resources, and then work backwards to understand your confidence level and margin of error; While you generally want to approach it from a statistical standpoint, sometimes it’s more practical to do it this way, and the tradeoff in accuracy is deemed acceptable

Data Collection & Management Systems Entry Organization Cleaning Validation Analysis Dissemination Feedback

Data Collection Systems Off-the-Shelf Applications Pros: Quick to implement Cons: Rigid in structure; often do not allow for flexibility Variable in cost Custom Applications Pros: Customized user interface; meets all data needs Cons: Requires infrastructure, programming, maintenance, support; needs may change over time High cost Hybrid Systems Pros: Customizable; flexible to changes in data needs Cons: Need in-house programming expertise Paper-based Systems Pros: Minimal technology requirements; cheap to design and implement Cons: Time-consuming; require extensive data entry Third-Party Contractors

Data Collection Costs Staffing (Training, Piloting, Collection, Analysis) Enumerators Programmers Analysts Logistics Vehicles, fuel Per Diem Equipment Mobile Devices Databases/Servers Software Data Collection Data Analysis

Thank you!