Electronic database searches were performed to update the 2006 systematic review for the time period January 2005 to March 2010, inclusive, searched on 27 May 2010. The end search date was purposely selected to exclude the date of publication of the most recent CONSORT Statement (CONSORT 2010) as it was not expected it to have been in use for a sufficient period to have produced evaluations.
MEDLINE (via OVID), EMBASE (via OVID), the Cochrane Methodology Register and the Cochrane Database of Systematic Reviews, both using the Wiley interface, were searched using a comprehensive strategy . The following citation indices were also searched using the ISI Web of Knowledge interface: The Science Citation Index, Social Science Citation Index, and Arts and Humanities Citation Index. Electronic records were stored and managed online through systematic review data management software, DistillerSR® .
Studies were included in the review if they evaluated the completeness of reporting of RCTs, and could be included in any of the following comparison groups, by assessing the: (1) completeness of reporting assessed in RCTs published in CONSORT-endorsing journals compared to non-endorsing journals; (2) completeness of reporting of RCTs published in CONSORT endorsing journals before and after endorsement; or (3) completeness of reporting of RCTs before and after the publication of the 1996 or 2001 CONSORT Statement. Studies were not excluded based on language of publication or validity assessment. The CONSORT checklist is being endorsed by non-medical journals, typically in veterinary trials , such ‘non-human’ trial evaluations were not eligible for this review.
When the eligibility of studies was either unclear or the data reported were insufficient to categorize the study into at least one of the reviews’ pre-specified comparison groups, corresponding authors were contacted for clarification and invited to submit unpublished data. This enabled judgment about study eligibility and, if eligible, provided data which were extracted for meta-analyses, when possible. For example, if the citations for RCTs included in an evaluation were provided, review authors could determine endorsement status of their publishing journals at the time of RCT publication by checking a journal’s Instructions to Authors and/or contacting editors of journals. A journal was considered an endorser if it had endorsed CONSORT at least months prior to publication of the evaluated RCT.
Title, abstract and full text screening of potentially relevant records was completed by two independent reviewers. One reviewer extracted general study characteristics of included studies, with complete verification by a second reviewer. Data on the completeness of reporting were extracted by one reviewer; a second reviewer verified accuracy of a 10% random sample. Discrepancies were resolved by discussion between the two reviewers or arbitration with a third member of the research team, when necessary. No modifications to the extracted data were made post-verification. Validity of included studies was assessed by one reviewer with complete verification by a second reviewer using pre-specified criteria .
Data on 27 outcomes were collected to estimate completeness of reporting of RCTs in included evaluations. These were: any of the 22 items on the 2001 CONSORT checklist, any of four additional sub-items for blinding (outcome assessor, intervention, patients and data analyst), or ‘Total Sum Score’ (that is, an aggregate score of some or all CONSORT checklist items). While data extraction was based on items of the 2001 CONSORT checklist, where studies evaluated the 1996 checklist, analyses were sub-grouped by checklist year since some checklist items were known to have undergone substantial modification in the 1996 to 2001 CONSORT update. These items were: ‘Title and Abstract’, ‘Outcomes’, ‘Sample Size’, ‘Participant Flow’ and ‘Numbers Analysed’ .
Relative proportions of RCTs adequately reporting any of the 27 outcomes were extracted from included studies. In addition, if quality of RCTs within included evaluations was assessed using a quality assessment tool however measured (for example, Jadad scale ), these data were extracted.
Relative risks (RR) and 99% confidence intervals were calculated to determine effect estimates for each outcome across evaluations. Reported mean scores across some or all checklist items, described as a ‘Total Sum Score’ were also pooled where a standardised mean difference (SMD) greater than zero indicates more complete reporting among RCTs published in CONSORT-endorsing journals.