1. Blowing in the Wind: The Quest for Accurate Crop Variety Identification in Field Research, with an Application to Maize in Uganda
TALIP KILIC
Senior Economist & Survey Methods Team Leader
Living Standards Measurement Study
Development Data Group – Survey Unit – World Bank
Co-Authors: JOHN ILUKOR, JAMES STEVENSON, SYDNEY GOURLAY, FREDERIC KOSMOWSKI, ANDRZEJ KILIAN, JULIUS PYTON SSERUMAGA, AND GODFREY ASEA
International Consortium on Applied Bioeconomy Research (ICABR) Conference
Berkeley, CA – May 31, 2017

2. Motivation
Accurate identification of crop varieties grown by farmers key to estimating
levels of improved variety cultivation
ensuing impacts on production, productivity, and a range of welfare and nutrition outcomes
Empirical evidence central to justifying investments in crop R&D, support to seed systems
Among farmers, correct information essential to their adoption & management decisions

3. Prevailing Approaches to Variety Identification
Extent of underinvestment in methodological innovation for accurate variety identification is puzzling
Literature on adoption & impacts have usually relied on expert estimates and/or farmer-reported survey data on
Variety names
Improved vs. traditional status of a cultivated variety
Hybrid vs. OPV status of a cultivated variety
Why worry?
Weaknesses in extension & formal seed systems
Reliance on informal channels of seed acquisition
Variety naming systems that exhibit variation across time & space
Limited empirical evidence on the accuracy of prevailing approaches to variety identification (& implications of measurement error in impact evaluation)
Farmer-elicitation method may work if the cultivated seeds are purchased from a formal seed system that truthfully label purchased seed attributes or that does not suffer from widespread seed adulteration and/or counterfeiting.
Given, however, the well-documented weaknesses in extension and seed systems, and the reliance on informal methods of seed acquisition, farmers may misidentify or not be able to identify the varieties they plant, and fail to provide the minimum set of correct varietal information sought in surveys.
The variety naming systems that emerge in the absence of formal seed systems and that exhibit variation across time and space is another complicating mediating factor in obtaining reliable information from survey respondents.

4. Our Contribution
Implemented a survey experiment in Eastern Uganda to test the relative accuracy subjective approaches to maize variety identification compared to DNA fingerprinting
Compiled a reference library of improved varieties in Uganda that serves a key input into DNA fingerprinting as well as the assessment of commercial seed quality

5. MAPS: Methodological Experiment on Measuring Maize Productivity, Soil Fertility, and Variety
Support
LSMS Minding the (Agricultural) Data Gap Research Program, funded by UK Aid
Global Strategy to Improve Agricultural and Rural Statistics, housed at FAO
World Bank Innovations in Big Data and Analytics Program
World Bank Trust Fund for Statistical Capacity Building
Primary Objectives
Test subjective approaches to measurement vis-à-vis objective methods for maize yield measurement, soil fertility assessment & maize variety identification
Partnerships
Uganda Bureau of Statistics (Implementing Agency), World Agroforestry Centre (Soil Fertility), CGIAR Standing Panel on Impact Assessment (Variety Identification), Stanford University & Terra Bella (Remote Sensing)
Round I (First Agricultural Season of 2015)
Post-Planting Fieldwork: April-June 2015
Crop Cutting Fieldwork: June-August 2015
Post-Harvest Fieldwork: September-November 2015
Round II (First Agricultural Season of 2016)
Identical timeline & visit structure
Follow-up to a subset of Round I households (540 out of 900)

6. MAPS Sample Round I Enumeration Area (EA) Selection
45 EAs from a 400 Km2 remote sensing tasking area (Iganga & Mayuge)
15 EAs in each of Serere & Sironko districts
Household Selection
Original Plan: 6 pure stand & 6 intercropping households selected at random in each EA following listing – 450 in each universe
Result: 385 vs. 515 split – inadequate # of pure stand HHs in select EAs
249 vs. 291 split in Iganga & Mayuge
Plot Selection
Survey Solutions CAPI application to randomly select one plot per household
Round II
Follow-up to 540 households in Iganga & Mayuge
Analysis sample: 440 households with crop cuts in Round I & II
Attrition does not have a bearing on the analysis
Note - attrition

7. MAPS Remote Sensing Tasking Area

8. MAPS Methods Methods Tested: Maize Production Crop-cutting
4m x 4m & a 2m x 2m subplot in Round I
8m x 8m sub-plot in Round II
Full-plot crop cut in Round II (1/2 of sample)
Remote sensing based on high-res imagery
First in testing the method in a smallholder production system against an objective measure
Self-reported harvest
Conversion of quantities in non-standard unit-condition combos into KG-, dried grain terms (“official” methods)
Land Area
GPS measurement (Garmin eTrex 30 handheld units)
Self-reported area
Soil Fertility
(Round I)
Conventional Soil Analysis (subsample)
Spectral Soil Analysis
Self-reported soil quality & attributes
Variety Identification
DNA fingerprinting of grain sampled from the crop-cutting subplot harvest (4x4m in Round I, 8x8m in Round II)
Self-reported variety name, type & morphological attributes
Note on 4x4m vs. 2x2m (lack of) difference in Round 1, and (randomly selected) 4x4m (quadrant) vs. 8x8m (lack of) difference in Round 2

9. DNA Fingerprinting
Diversity Arrays Technology (DArTseq) method that facilitates genome-wide characterizations of large accessions sets compared to existing genotyping-by-sequencing methods using SNP markers
Compiled a reference library of 38 maize varieties in circulation during the pre-planting period of the first rainy season of 2015, from NARO & 4 major seed companies, with revealed genotyping intention
Genotyped each reference library and field samples to derive two vars
Heterogeneity: # of DNA marker variants in the genomic representation - a collection of fragments from the genome selected for sequencing
Purity: Computed only for the field samples, represents the extent to which heterogeneity overlaps with that of the matched reference library variety - identified initially as the one with the closest genetic distance to the field sample in question (below a distance threshold of 3).

10. Recursive Partitioning & Classification Tree Analysis of Morphological Attributes of 38 Reference Library Samples
Morphological attributes for the reference library: Obtained by planting out the 38 varieties in NaCCRI fields.
Results: Varieties are uniquely identified using 11 attributes.
Identification of the varieties in the field: Using these attributes, varieties that the farmers plant were identified based farmer responses on morphological attributes
The feasibility and the accuracy of this approach are yet to be tested especially since morphological characteristics (1) are often multi-genic and are not available at all growth stages, and (2) are not stable and may reflect an adaptation to environmental conditions, particularly in the case of maize.
The first step in RPCTA is to find an explanatory variable that best splits the data into two groups. This process is repeated in each sub-group, and recursively, until the subgroups either reach a minimum size or until no improvement can be made. The second step is to trim back the full classification tree based on cross-validation (Therneau et al., 2015).

11. Context
83.9 percent of the population live in rural areas. National rate of poverty = 19.5 percent.
Agriculture value added corresponds to 25.8 percent of the GDP. Agricultural employment makes up 71.7 percent of total employmen.
Maize is one of the major staple, commercial, and export crops in Uganda. It is the leading cereal crop grown in almost all parts of the country. In Eastern Uganda; the country’s leading maize producing region, the crop accounts for the highest share (25 percent) of crop income. At the same time, Eastern Uganda, following Northern Uganda, is also the region with the highest concentration of the country’s poor, and the latest estimate of the regional absolute poverty rate stands at 24.5 percent.
While not shown, the sample descriptive statistics are comparable to those derived for the Eastern region, and Iganga and Mayuge sub-samples of the UNPS 2015/16.
The average area was 0.14 hectares for our plots, 46 percent of which was monocropped with maize. The sampled plots were all within a 1 kilometer radius of the dwelling units.
The non-labor input use was low (e.g. the incidence of any inorganic fertilizer use was only 9 percent), and the lion share of the plot-level labor input originated from within households.
Forty-two percent of the sampled plots were managed by females, and the plot managers were, on average, 41 years old, with slightly above 6 years of education.
Regarding specifically the maize grown within the crop cut sub-plots, the seed for 37 percent were acquired commercially in the planting period for the agricultural season of interest. Further, only 45 percent of the farmers could provide a name for the primary variety grown on the plot selected for crop cutting.

12. How Do Different Methods Perform in Unique Identification of Maize Varieties in Round I?
53 percent of farmers could not state the variety they have planted
Farmer-reported morph. attributes does not uniquely identify varieties
DNA fingerprinting performs the best for unique varietal identification

13. Only 2 Percent of the Farmers Correctly Identified the
Variety Based on DNA Analysis in Round I
With the exception of LONGE 10H, the varieties stated by the farmers (i.e. right panel) are NOT among the varieties identified by DNA fingerprinting (i.e. left panel).
Either farmers do not know or the stated names are the ones they were told
Source: Ilukor et al. (Forthcoming).

14. And Our Experts Were No Better!

15. How Do Different Methods Perform in Identifying Local/Improved & OPV/Hybrid Varieties in Round I?
Cultivation of improved & hybrid varieties is under-estimated by farmers
Cultivation of open pollinated varieties is over-estimated by farmers

16. Purity of the Field Samples in Round I
Farmer-planted variety according to DNA fingerprinting is the reference library variety that is genetically closest (not necessarily identical)
Purity = Overlap between the field sample genetic heterogeneity & the genetic heterogeneity of the identified variety in the reference library
Mean
63.2%
Median
62.1%
Min
46.9%
Max
98.4%

17. Headline Findings from Multivariate Analyses
of Variety Identification Outcomes
Purity
Negatively correlated with farmer’s correct identification of recyclability of the seed
NO relationship with commercial acquisition of the seed!
Farmer’s correct identification of an improved variety
Positively correlated with farmer’s knowledge of the variety & commercial acquisition of the seed

18. (Unacceptable Levels of) Heterogeneity in
Reference Library Samples in Round I
Acceptable level of heterogeneity of the samples is 20% but most of the reference library samples are above the threshold.
Mean
32.9%
Median
24.6%
Min
9.8%
Max
75.2%

19. Key Take-Away Messages
Variety identification findings reveal:
High-levels of improved variety cultivation, despite popular belief
But… cultivated varieties are of inferior quality
Limited farmer knowledge about the varieties that they plant
Weaknesses in & potential implications for extension & seed system
Evidence prompts us to think more critically about existing agricultural statistics & survey methods
Support for DNA fingerprinting to be the new standard for accurate variety identification in field research
Further experimentation & synthesis of evidence from completed survey experiments on other countries & crops – key to formulating guidelines for scale-up
Additional costs require more thinking around sub-sampling approaches in existing household & farm surveys

20. Blowing in the Wind: The Quest for Accurate Crop Variety Identification in Field Research, with an Application to Maize in Uganda
TALIP KILIC
Senior Economist & Survey Methods Team Leader
Living Standards Measurement Study
Development Data Group – Survey Unit – World Bank
Co-Authors: JOHN ILUKOR, JAMES STEVENSON, SYDNEY GOURLAY, FREDERIC KOSMOWSKI, ANDRZEJ KILIAN, JULIUS PYTON SSERUMAGA, AND GODFREY ASEA
International Consortium on Applied Bioeconomy Research (ICABR) Conference
Berkeley, CA – May 31, 2017