An Examination of Literature Searching Methodologies in Crop-Related Meta-Analyses Brad Brazzeal Agriculture & Forest Resources Librarian Mississippi State University

What is a meta-analysis? “… the statistical analysis of the results of multiple independent studies, usually from the published and/or gray literature, to estimate the magnitude, consistency, homogeneity of an effect of interest” (Sherm et al., 2014)

Importance of meta-analysis in agricultural research “Meta-analysis would make it possible to quantify variations of cropping system performances in interaction with soil and climate conditions more accurately across environments and socio-economic contexts” (Dore et al., 2011)

Providing literature search details is vital Philibert et al. (2012) emphasize the need for …  “Correct description of the bibliographic search procedures used by the authors to select individual studies …”  “Listing of the references of the selected individual studies used in the meta-analysis” Koricheva and Gurevitch (2014) expand on Philbert’s first point …  “Are details of full bibliographic search (electronic data bases [sic] used, keyword combinations, years) reported in sufficient detail to allow replication?”

Questions addressed in this study  Which databases are used when finding studies to include?  Is enough information given for others to replicate the searches?  Are non-journal articles typically included?  Are clear lists of included studies provided?

Methods  Used the following search string in CAB Abstracts ( "crop production" OR "crop yield" OR "crop management" OR "cropping systems" ) AND "meta- analysis" NOT ( QTL OR "quantitative trait loci" )  Limited to academic journal articles in English published from

Methods (contd.)  Excluded articles that were (1) not a meta-anlaysis, (2) not based on publications, or (3) that just used previous publications from the authors  Each was evaluated for the following:  Databases authors used to find studies to include  Details of search strategy  Inclusion of non-journal publications  Presence of a clear listing of studies included

Results  117 articles from 57 journals met the criteria  All but 4 were from JCR journals  19 results from Agriculture, Ecosystems and Environment

Databases Used Database NameStudies that Used It Science Citation Index / Web of Science / Web of Knowledge 61% Google Scholar22% China National Knowledge Infrastructure (CNKI) / China Journal Net (CJN) 10% Scopus8% ScienceDirect7% CAB Abstracts5% Other sources mentioned 2 or 3 times: Agricola, AgEcon Search, FAO, JSTOR, RePEc, Springer

Databases Used (contd.)  Other sources mentioned 2 or 3 times: Agricola, AgEcon Search, FAO, JSTOR, RePEc, Springer  25% also searched reference lists to find studies  25% gave no indication of databases searched

Details of search strategy ItemStudies Search terms complete enough to replicate34% Date of search / range of years given56% Both of the above26%* * By comparison, 22% of meta-analyses met the “repeatable procedure” criterion in Philibert et al. (2012), and 32% of meta- analyses that gave “complete details of bibliographic searches in Koricheva and Gurevitch (2014).

Inclusion of non-journal publications Included Non-Journal Pubs?Studies Yes62% No13% Unclear26%

Clear listing of studies included  90% of the meta-analyses included a list of studies  Comparable to the 92% of meta-analysis that met the “references” criterion in Philibert et al. (2012)

Conclusion  Many crop-related meta-analyses do not provide sufficient information to replicate searches.  Researchers wishing to conduct a meta-analysis could benefit from the expertise of librarians.

References  Doré, T., Makowski, D., Malézieux, E., Munier-Jolain, N., Tchamitchian, M., & Tittonell, P. (2011). Facing up to the paradigm of ecological intensification in agronomy: Revisiting methods, concepts and knowledge. European Journal of Agronomy, 34(4),  Koricheva, J., & Gurevitch, J. (2014). Use and misuses of meta-analysis in plant ecology. Journal of Ecology 102:  Philibert, A., Loyce, C., & Makowski, D. (2012). Assessment of the quality of meta- analysis in agronomy. Agriculture, Ecosystems and Environment, 148,  Scherm, H., Thomas, C. S., Garrett, K. A., & Olsen, J. M. (2014). Meta-analysis and other approaches for synthesizing structured and unstructured data in plant pathology. Annual Review of Phytopathology 52: