Table 3 Handsearching results

Adams et al. [25]

9% (67 out of 698) of RCTs (CI 7–11%; Sensitivity 94% (CI 93–95%);Precision 7% (CI 6–8%).

▪ Conference abstracts and letters not indexed in databases;

▪ RCTs not indexed, or no methodological data available to identify studies;

▪ Methodological descriptors (i.e. ‘random’ for allocation) were overlooked by database indexers.

Standard MEDLINE search: sensitivity 18% (CI 15–21%) Precision 40% (CI 35–45%)

Skilled MEDLINE searched: Sensitive: 52% (CI 48–56%) Precision 59% (CI 55–65%)

▪ Studies missed by searcher error/fatigue;

▪ Methodological data being ‘hidden’ in article

Armstrong et al. [31]

6 out of 131 (4.6%) RCTs/CCTs

▪ Trials made no reference in abstract, title or subject headings to random allocation;

▪ Trials used terms for random allocation it the title, abstract or MeSH but were not correctly indexed by publication type;

▪ Trials were abstracts;

▪ Studies were identified in supplement editions of journals not indexed in MEDLINE; and

▪ Not found in MEDLINE as issue appeared missing in MEDLINE.

125 (of 131) studies would have been identified by a MEDLINE using PICO search. 118 (of 131) would have been identified by a PICOs search

Not reported

Blümle and Antes[32]

10, 165 RCTs/CCTs out of 18,491(55%)

▪ Incorrect indexing and incomplete compilation of health care journals in electronic databases impair result of systematic literature search.

Not reported in abstract

Not reported in abstract

Croft et al. [33]

7 out of 10 (70%)

▪ Two RCTs identified through letter to editors

▪ Not picked up in MEDLINE search

3 studies identified in MEDLINE (30%)

Not reported

Glanville et al. [26]

7 out of 25, although none of these studies met the review’s inclusion criteria.

Not reported

Electronic searching (including reference checking), by comparison, yielded 30 included papers.

Not reported

Hay et al. [26]

5 of 40 studies identified (compared to EMBASE) or 13 of 40 (compared to PsycLIT)

Not reported

EMBASE n = 35 (out of 40) RCTs (88%) and precision 9%.

PsycLIT n = 27 (out of 40) and precision 9%.

EMBASE:

▪ n = 3 journal years not indexed

▪ n = 2 reason unclear.

PsycLIT:

▪ n = 13 gap in indexing and current material being loaded.

Hopewell et al. [27]

369 out of 714 (52%) RCTs

▪ 252/369 (68%) no MEDLINE record.

▪ 232/252 (92%) abstracts and/or published in supplements.

32 of 714 (4%)

Jadad et al. [28]

Handsearching vs MEDLINE 25 out of 151 (16.5%)precision 2.7%handsearching vs supplement editions and MEDLINE. 150 out of 162 eligible (precision 5.6%).

▪ Non-indexed abstract (n = 7);

▪ Non-indexed letter (n = 1);

▪ Search term random not in MeSH or abstract summary (n = 9);

▪ Key search term not in MeSH or in abstract summary (n = 7); and

▪ No apparent reason (n = 1).

handsearching vs MEDLINE 2 of 245 (0.8%) handsearching vs supplement editions and MEDLINE. 1 out of 13 (7.6%)

▪ Why studies were missed by handsearching is not reported or explored

Langham et al. [30]

227 out of 710 (32%)

Not reported

MEDLINE identified 118 (16.6%) of studies missed by Handsearching.

Not reported

Mattioli et al. [35]

0 out of 25 (0), (all identified by handsearching)

Not reported

Specific PubMed Search

16 out of 25 (64%)

Sensitive PubMed Search

9 out of 25 (36%)

Not reported

Milne and Thorogood [36]

34 out of 82 (41.5%)

Not reported

Capture/recapture used to test. Estimated n = 3 missed by handsearching.

▪ Inadequate indexing or trials not indexed on MEDLINE

▪ Prohibits are not located by computerised searches