Skip to main content

Advertisement

Table 3 Handsearching results

From: A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review

  Advantages Disadvantages
Studies No. identified by handsearching but missed by MEDLINE Why studies were missed by MEDLINE (claimed advantages of handsearching) No. identified by MEDLINE but missed by handsearching Why studies were missed by handsearching (claimed advantages of database searching)
Adams et al. [25] 9% (67 out of 698) of RCTs (CI 7–11%; Sensitivity 94% (CI 93–95%);Precision 7% (CI 6–8%). ▪ Conference abstracts and letters not indexed in databases; ▪ RCTs not indexed, or no methodological data available to identify studies; ▪ Methodological descriptors (i.e. ‘random’ for allocation) were overlooked by database indexers. Standard MEDLINE search: sensitivity 18% (CI 15–21%) Precision 40% (CI 35–45%) Skilled MEDLINE searched: Sensitive: 52% (CI 48–56%) Precision 59% (CI 55–65%) ▪ Studies missed by searcher error/fatigue; ▪ Methodological data being ‘hidden’ in article
Armstrong et al. [31] 6 out of 131 (4.6%) RCTs/CCTs ▪ Trials made no reference in abstract, title or subject headings to random allocation; ▪ Trials used terms for random allocation it the title, abstract or MeSH but were not correctly indexed by publication type; ▪ Trials were abstracts; ▪ Studies were identified in supplement editions of journals not indexed in MEDLINE; and ▪ Not found in MEDLINE as issue appeared missing in MEDLINE. 125 (of 131) studies would have been identified by a MEDLINE using PICO search. 118 (of 131) would have been identified by a PICOs search Not reported
Blümle and Antes[32] 10, 165 RCTs/CCTs out of 18,491(55%) ▪ Incorrect indexing and incomplete compilation of health care journals in electronic databases impair result of systematic literature search. Not reported in abstract Not reported in abstract
Croft et al. [33] 7 out of 10 (70%) ▪ Two RCTs identified through letter to editors ▪ Not picked up in MEDLINE search 3 studies identified in MEDLINE (30%) Not reported
Glanville et al. [26] 7 out of 25, although none of these studies met the review’s inclusion criteria. Not reported Electronic searching (including reference checking), by comparison, yielded 30 included papers. Not reported
Hay et al. [26] 5 of 40 studies identified (compared to EMBASE) or 13 of 40 (compared to PsycLIT) Not reported EMBASE n = 35 (out of 40) RCTs (88%) and precision 9%. PsycLIT n = 27 (out of 40) and precision 9%. EMBASE: ▪ n = 3 journal years not indexed ▪ n = 2 reason unclear. PsycLIT: ▪ n = 13 gap in indexing and current material being loaded.
Hopewell et al. [27] 369 out of 714 (52%) RCTs ▪ 252/369 (68%) no MEDLINE record. ▪ 232/252 (92%) abstracts and/or published in supplements. 32 of 714 (4%)  
Jadad et al. [28] Handsearching vs MEDLINE 25 out of 151 (16.5%)precision 2.7%handsearching vs supplement editions and MEDLINE. 150 out of 162 eligible (precision 5.6%). ▪ Non-indexed abstract (n = 7); ▪ Non-indexed letter (n = 1); ▪ Search term random not in MeSH or abstract summary (n = 9); ▪ Key search term not in MeSH or in abstract summary (n = 7); and ▪ No apparent reason (n = 1). handsearching vs MEDLINE 2 of 245 (0.8%) handsearching vs supplement editions and MEDLINE. 1 out of 13 (7.6%) ▪ Why studies were missed by handsearching is not reported or explored
Langham et al. [30] 227 out of 710 (32%) Not reported MEDLINE identified 118 (16.6%) of studies missed by Handsearching. Not reported
Mattioli et al. [35] 0 out of 25 (0), (all identified by handsearching) Not reported Specific PubMed Search 16 out of 25 (64%) Sensitive PubMed Search 9 out of 25 (36%) Not reported
Milne and Thorogood [36] 34 out of 82 (41.5%) Not reported Capture/recapture used to test. Estimated n = 3 missed by handsearching. ▪ Inadequate indexing or trials not indexed on MEDLINE ▪ Prohibits are not located by computerised searches