Types of studies
Non-randomised studies include a comparison of two or more interventions to improve recruitment to randomised controlled trials. We define ‘non-randomised studies’ as any quantitative study estimating the effectiveness of a recruitment intervention that did not use randomisation to allocate participants to intervention or comparison groups. These types of studies are referred to by multiple names in the literature including but not limited to observational studies, cohort studies and case-control studies.
Studies covering questionnaire response and studies of retention strategies will be excluded on the grounds that these are separate issues addressed by their corresponding Cochrane Methodology reviews [12, 13].
Participants
Individuals involved in a trial. The context of the trial is likely to be healthcare but may not be, for the reason that interventions that are effective in other fields may also be applicable to settings in the healthcare environment. Strategies evaluated within simulated trials (studies that ask potential participants whether they would take part in a trial if it was to be run but the study does not run the trial) will not be eligible.
Types of intervention
Any intervention or approach aimed at improving or supporting recruitment of participants nested within studies performed for purposes unrelated to recruitment. Included interventions could be wide-ranging, aimed at research ethics committees (e.g. interventions supporting the case for non-requirement of mandatory signed and witnessed consent for recruitment to a trial), collaborators (e.g. healthcare professionals recruiting patients for a trial) or study participants (e.g. patients being randomised to a trial). Examples of such interventions are the use of SMS appointment reminders for participants supplying a mobile telephone number, use of newspaper or social media advertising, a dedicated email address or a toll-free telephone number as the initial point of contact from interested participants, simplified consent procedures for participants within a particular age range and recruitment coordinators working at selected trial sites.
Types of comparator
Any approach aimed at improving or supporting recruitment of participants nested within studies performed for purposes unrelated to recruitment. Examples are as given in the ‘Types of intervention’ section. The comparator could also be nothing, meaning the intervention is compared against taking no special measures to improve recruitment.
Types of outcome measures
Primary: Number of individuals or centres recruited into a randomised controlled trial.
Secondary: Cost of using the recruitment intervention per recruit.
Search methods for identification of studies
The search strategy was developed by an Information Specialist (CF), who has expertise in developing search strategies for health technology assessment, especially through being the Information Specialist attached to the Aberdeen-based National Institute of Health Research Technology Assessment Review Group (http://www.nets.nihr.ac.uk/programmes/hta/policy-makers/tar-teams). The strategy was reviewed by the other authors. To identify studies, we will search bibliographic databases and hand-search reference lists of both relevant systematic reviews and all included studies. We will search systematic reviews of randomised recruitment interventions (e.g. [9]) for studies that were excluded on the grounds that they were not randomised.
We will apply neither language nor time restriction.
We will search the following databases:
Cochrane Methodology Register (CMR)
Medical Literature Analysis and Retrieval System Online (MEDLINE), OVID 1946 to present
MEDLINE In-Process, OVID current
Excerpta Medica dataBASE (EMBASE), OVID 1947 to present
Cumulative Index to Nursing and Allied Health Literature (CINAHL), EBSCO 1981 to present
PsycINFO, Ovid 1967 to present
The following multifile search strategy for MEDLINE and EMBASE (OVID) will be adapted for the other databases listed.
-
1 *Patient selection/
-
2 *informed consent/
-
3 ((participat$ or recruit$ or enter$) adj4 (trial? or research or study or studies or random$)).tw.
-
4 (participat$ adj4 (recruit$ or decide$ or decision or agree$ or consent$)).tw.
-
5 1 or 2 or 3 or 4
-
6 exp *clinical trials as topic/use prmz
-
7 exp *“clinical trial (topic)”/use emcz
-
8 *research subjects/use prmz
-
9 *research subject/use emcz
-
10 recruit$.ti.
-
11 informed consent.ti.
-
12 (recruit$ adj7 (trial? or research or study or studies)).ti.
-
13 (participa$ adj7 (trial? or research or study or studies)).ti.
-
14 or/6–13
-
15 5 and 14
-
16 controlled clinical trial.pt. use prmz
-
17 controlled clinical trial/use emcz
-
18 intervention studies/
-
19 randomized controlled trials as topic/use prmz
-
20 “randomized controlled trial (topic)”/
-
21 experiment$.tw.
-
22 quasi-experimental study/ use emcz
-
23 impact.tw.
-
24 intervention?.tw.
-
25 chang$.tw.
-
26 evaluation studies/
-
27 evaluat$.tw.
-
28 effect$.tw.
-
29 examin$.tw
-
30 compar$.tw.
-
31 comparative studies/
-
32 or/16–31
-
33 exp animals/ not humans/
-
34 32 not 33
-
35 15 and 34
-
36 35 not (randomized controlled trial.pt. or randomi?ed.ti.
-
37 36 not randomized controlled trial/use emcz
-
38 37 not (letter or editorial or comment or abstract).pt.
Data management
All search results will be merged into the reference management software EndNote, and duplicate records of the same report will be removed using the EndNote de-duplication tool. We will use a master spreadsheet to track all inclusions/exclusions, which will allow us to create a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram once the screening process is complete (Additional file 1). Extracted data will be collected on specially designed forms (see ‘Data extraction’ section) and data entered into Cochrane RevMan tool (http://tech.cochrane.org/revman) when the data make this possible.
Identifying studies
Two authors will independently screen the titles and abstracts of all records retrieved from searches of the electronic bibliographic databases stated above. The full text will be acquired for studies that look as if they meet the inclusion criteria. The full texts of all potentially eligible studies will be independently reviewed by two authors to determine if they meet the stated inclusion criteria. We will seek additional information from study authors where necessary to resolve questions about eligibility and provide other data as required. Any disagreements throughout the process of trial identification and selection will be resolved through discussion and, if necessary, the involvement of a third reviewer.
Risk of bias of individual studies
As we are using non-randomised studies only, the likelihood of increased heterogeneity resulting from residual confounding is heightened. The Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I; https://sites.google.com/site/riskofbiastool/) will be used to assess the risk of bias due to confounding. Aspects of methodological quality such as participant selection, measurement of intervention, departures from intended interventions, missing data, measurement in outcomes and selection of the reported result in included non-randomised studies will be assessed using the ROBINS-I tool [14] (see https://sites.google.com/site/riskofbiastool/). Each study will be rated as critical, serious, moderate or low risk of bias based on a judgement of the gathered information. If there is insufficient detail reported in the study, the risk of bias will be classified as ‘no information’ and the original study authors will be contacted for more information. Reporting of information on the flow of participants through the trial (e.g. from a Consolidated Standards of Reporting Trials (CONSORT) diagram) will be recorded.
Data on risk of bias will be presented in an additional table for all included studies, and results will be interpreted in light of risk of bias; studies will not be excluded on the grounds of risk of bias. Where it is appropriate to do a meta-analysis, risk of bias will be included in the GRADE assessment of study limitations.
Data extraction
A data extraction form will be developed to collect the outcome measures given under the ‘Types of outcome measures’ section. Data will be extracted from each article by two authors independently. Disparities in data extraction will be resolved through discussion and, if necessary, the involvement of a third reviewer. We will contact the authors of reports of potentially relevant studies to try and obtain information or additional data needed for the review that is not available in the published reports. Data will be extracted on the recruitment intervention evaluated, country in which the study was conducted, type of population, details on the study setting, description of study to be recruited into and numbers and proportions recruited using each intervention and comparator.
Data synthesis
Trials will be analysed according to the type of intervention used in the study (e.g. newspaper articles, use of SMS reminders); interventions will be grouped when their form or content is deemed sufficiently alike. A further categorisation by type of participant will be used if we find the same intervention applied to more than one type of participant (e.g. patients, staff at recruiting centres).
Dealing with missing data
Attempts will be made to contact study authors to obtain any missing data (e.g. participant, intervention or outcome details). Analyses will be conducted on an intention-to-treat basis where possible; alternatively, data will be analysed as reported. Loss to follow-up will be reported and assessed as a potential source of bias in our risk of bias assessment.
Assessment of heterogeneity
The likely nature of the included studies means that we anticipate that much of the analysis will be narrative description of the data rather than a statistical analysis, which we intend to present in tables, grouped by intervention type (and possibly participant if the same intervention is used with different types of participant). Where population, intervention and outcome are sufficiently similar to allow pooling of data in a meta-analysis, we will look for both visual evidence of heterogeneity in forest plots and statistical evidence of heterogeneity using the chi-square test for heterogeneity and the degree of heterogeneity quantified using the I
2 statistic [14]. Where substantial heterogeneity is detected (I
2 ≥ 50 %), possible explanations will be investigated informally and the data summarised using a random-effects model if appropriate.
Assessment of reporting bias
We will investigate reporting (publication) bias for the primary outcome using a funnel plot where 10 or more studies of the same population, intervention and outcome are available. Care will be taken when interpreting any asymmetry in the funnel plots as this is not always due to publication bias.
Confidence in cumulative estimate
Where possible, we will pool studies and apply the GRADE approach to give an overall assessment of the certainty of the evidence [10, 15]. Certainty will be considered either as high (further research is very unlikely to change our confidence in the estimate of effect), moderate (further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate), low (further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate) or very low (very uncertain about the estimate of effect). Two authors will independently apply GRADE to assess the certainty of the evidence.