Public Defence: Mariann Mathisen

Introduction:

Systematic reviews aim to find and synthesize all relevant research as a basis for evidence based practice. The methods used in systematic reviewing, however, are extremely time and resource demanding. More efficient methods should be tested to make better use of scarce resources, and this study has examined three main approaches to simplify and optimize the search process for systematic reviews of healthcare interventions.

Methods:

First, a cross sectional study was performed to investigate whether searching MEDLINE and Embase was enough for study identification in systematic reviews with respect to database coverage and search performance in a sample of 400 Cochrane reviews. Next, an exploratory design was used to find the optimal combination of search terms in a subsample of 254 of the sample reviews including 5 to 50 primary studies. Simpler search strategies containing the optimized search terms were developed and tested in Ovid MEDLINE and PubMed. Last, to test an automated information retrieval method for systematic reviews, the ranking function in PubMed was tested on conventional Boolean search strategies. Three scoping reviews were performed to determine what research exists on the main approaches and to inform the empirical studies.

Results:

MEDLINE alone had a coverage of 84 % and MEDLINE and Embase combined 90 % in the sample of 400 reviews. The published Ovid MEDLINE search strategies of the sample reviews had 88 % recall and 2.8 % precision. In the subsample of 254 reviews, coverage was 85 % for MEDLINE alone and 90 % for MEDLINE and Embase combined. Recall was 87 % and precision 2.8 % for the search strategies of the subsample, which means that number of included studies in the sample reviews did not influence the total average values to any degree for this sample. An average of 7.2 search terms in each Ovid MEDLINE search strategy was necessary to retrieve the included studies of the subsample. That is approximately one tenth of the number of search terms used in the published search strategies. An average of 3.0 search terms per review occurred in the review title and 2.8 in the rest of the protocol. Average MAP for all reviews was 7.2 % for the ranked Boolean searches in PubMed, which were higher than for other sort options, but recall decreased at almost all cut-off values.

Conclusions:

A high percentage of the included studies of the sample reviews in this study was indexed in MEDLINE and Embase, and the published Ovid MEDLINE search strategies had reasonably high performance in most cases. This means that it could be enough to search only these two databases for some topics if other sources were searched in addition, and for some review groups, searching only MEDLINE could be enough. Based on the results of this study, conventional Boolean search strategies seem to be unnecessary long and complicated with an exaggerated use of search terms compared to what is necessary to retrieve relevant studies. For the sample of this study, most search terms could be found in the review title and/or rest of the protocol, except for reviews in Effective Practice and Organization of Care Group (EPOC) and other complex reviews. Ranking search results using “Best Match” in PubMed did not succeed in moving all or most relevant studies towards the top of the result lists and is therefore not recommended to simplify the search process for systematic reviews.