Аннотация
Introduction. This paper provides an overview of the information retrieval strategy employed for two meta-analyses, conducted by a systematic review team at Concordia University (Montreal, QC, Canada). Both papers draw on standards first articulated by H.M. Cooper and further developed by the Campbell Collaboration, which promote a comprehensive approach to systematically searching an extensive array of resources (bibliographic databases, print resources, citation indices, etc.) in order to locate both published and unpublished research. The goal is to verify if searching comprehensively through multiple resources retrieves studies that are unique, and hence, improve the overall representativeness of a diverse body of literature. We also analyze the sensitivity and specificity of the results by data source.
Methods. In order to determine the source sensitivity, we consider percentage of results from each source retrieved for full-text review. In order to determine the source specificity, we derive a percentage from the total number of studies included in the final meta-analysis compared against the overall number of initial results found.
Results. Results demonstrate the need to search beyond the subject-specific databases of a particular discipline as unique results can be found in many places. Databases for related disciplines provided 129 unique includes to each meta-analysis, and multidisciplinary databases provided 44 and 99 unique includes for the two meta-analyses in question respectively. Manual search techniques were much more sensitive and specific than electronic searches of databases and yield a higher percentage of final includes.
Discussion. The results demonstrate the utility of a comprehensive information retrieval methodology like that proposed by the Campbell Collaboration, which goes beyond the main subject databases to locate the full range of information sources, including grey literature.
Библиографические ссылки
Schmid R. F., Bernard R. M., Borokhovski E., Tamim R. M., Abrami P. C., Surkes M. A., Wade C. A., Woods J. The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education, 2014, V. 72, pp. 271–291. DOI: 10.1016/j.compedu.2013.11.002
Cooper H. M. The integrative research review: A systematic approach. Thousand Oaks, CA, Sage Publications, 1984. 144 p.
Cooper H. M. Synthesizing research: A guide for literature reviews. 5th ed. Thousand Oaks, CA, Sage Publications, 2017.
Hammerstrøm K., Wade A., Jørgensen A.-M. K. Searching for studies: A guide to information retrieval for Campbell systematic reviews. Oslo, The Campbell Collaboration, 2010.
Kugley S., Wade A., Thomas J., Mahood Q., Jørgensen A.-M. K., Hammerstrøm K. T., & Sathe N. Searching for studies: A guide to information retrieval for Campbell systematic reviews. Oslo, The Campbell Collaboration, 2017. DOI: 10.4073/cmg.2016.1
Glass G. V. Primary, Secondary and Meta-Analysis of Research. Educational Researcher, 1976, V. 5, Issue 10, pp. 3–8. DOI: 10.3102/0013189X005010003
Glass G. V., McGaw B., & Smith M. L. Meta-analysis in social research. Beverly Hills, CA, Sage Publications, 1981.
Lipsey M. W., Wilson D. B. Practical meta-analysis. Thousand Oaks, CA, Sage, 1990.
Borenstein M., Hedges L., Higgins J., & Rothstein H. Introduction to meta-analysis. Chichester, UK, Wiley, 2009.
Lefebvre C., Manheimer E., & Glanville J. Searching for studies. In J. P. T. Higgins & S. Green (eds.) Cochrane Handbook for Systematic Reviews of Interventions (Version 5.1.0, Chap. 6). The Cochrane Collaboration, 2011. Retrieved from http://handbook.cochrane.org
Cooper H., Hedges L. V., & Valentine J. C. (eds.) The Handbook of Research Synthesis and Meta-Analysis (2nd ed.). New York, Russell Sage Foundation, 2009. 632 p.
Pettigrew M., & Roberts H. Systematic Reviews in the Social Sciences: A Practical Guide. Malden, MA, John Wiley & Sons, 2008.
Auger C. P. Use of reports literature. London, Butterworths, 1975. 226 p.
Auger C. P. Information Sources in Grey Literature (4th ed.). Boston, MA, Walter de Gruyter GmbH, 2017. 177 p.
Conn V. S., Valentine J. C., Copper H. M., & Rantz M. J. Grey Literature in Meta-Analyses. Nursing Research, 2003, V. 52, Issue 4, pp. 256–261.
Polanin J. R., Tanner-Smith E. E., & Hennessy E. A. Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review. Review of Educational Research, 2016, V. 86, Issue 1, pp. 207–236. DOI: 10.3102/0034654315582067
Rothstein H. R., Sutton A. J., & Borenstein M. (eds.) Publication bias in meta-analysis – prevention, assessment and adjustments. Chichester, UK, Wiley, 2005. 356 p.
Swinkels A., Briddon J., & Hall J. Two physiotherapists, one librarian and a systematic review: collaboration in action. Health Information and Libraries Journal, 2006, V. 23, Issue 4, pp. 248–256. DOI: 10.1111/j.1471-1842.2006.00689.x
Harris M. R. The librarian’s roles in the systematic review process: a case study. Journal of the Medical Library Association, 2005, V. 93, Issue 1, pp. 81–87.
DeLuca J. B., Mullins M. M., Lyles C. M., Crepaz N., Kay L., & Thadiparthi S. Developing a Comprehensive Search Strategy for Evidence Based Systematic Reviews. Evidence Based Library and Information Practice, 2008, V. 3, № 1, pp. 3–32. DOI: 10.18438/B8KP66
Helmer D., Savoie I., Green C., & Kazanjian A. Evidence-based practice: extending the search to find material for the systematic review. Bulletin of the Medical Library Association, 2001, V. 89, Issue 4, pp. 346–352.
Booth A. “Brimful of STARLITE”: toward standards for reporting literature searches. Journal of the Medical Library Association, 2006, V. 94, Issue 4, pp. 421–429.
Zhang L., Sampson M., & McGowan J. Reporting of the Role of the Expert Searcher in Cochrane Reviews. Evidence Based Library and Information Practice, 2006, V. 1, № 4, pp. 3–16. DOI: 10.18438/B85K52
Golder S., Loke Y., & McIntosh H. M. Room for improvement? A survey of the methods used in systematic reviews of adverse effects. BMC Medical Research Methodology, 2006, 6 (3). DOI: 10.1186/1471-2288-6-3
Yoshii A., Plaut D. A., McGraw K. A., Anderson M. J., & Wellik K. E. Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association, 2009, 97 (1), pp. 21–29. DOI: 10.3163/1536-5050.97.1.004
Wade C. A., Turner H. M., Rothstein H. R., & Lavenberg J. G. Information retrieval and the role of the information specialist in producing high-quality systematic reviews in the social, behavioural and education sciences. Evidence & Policy, 2006, V. 2, № 1, pp. 89–108. DOI: 10.1332/174426406775249705
Bernard R. M. Things I Have Learned about Meta-Analysis Since 1990: Reducing Bias in Search of “The Big Picture”. Canadian Journal of Learning and Technology, 2014, V. 40, № 3. DOI: 10.21432/T2MW29
Bernard R. M., Borokhovski E., Schmid R. F., & Tamim R. M. An exploration of bias in meta-analysis: The case of technology integration research in higher education. Journal of Computing in Higher Education, 2014, V. 26, Issue 3, pp. 183–209. DOI: 10.1007/s12528-014-9084-z
Bernard R. M., Borokhovski E., & Tamim R. M. Detecting bias in meta-analyses of distance education research: big pictures we can rely on. Distance Education, 2014, V. 35, Issue 3, pp. 271–293. DOI: 10.1080/01587919.2015.957433
Watson G., & Glaser E. M. Watson-Glaser critical thinking appraisal. San Antonio, TX, PsychCorp, 1980.
Ennis R. H., & Millman J. Cornell critical thinking test. Pacific Grove, CA, Critical Thinking Books & Software, 1985.