Reporting quality of Cochrane systematic reviews with Chinese herbal medicines

Background Chinese herbal medicines (CHMs) are the major interventions of traditional Chinese medicine (TCM), which are typically administered as either single herbs or formulas. The Cochrane systematic reviews (SRs) of CHMs are essential references for evaluating the efficacy and safety of CHMs interventions; they are expected to be accurate and reliable. This study aimed to assess the reporting quality of these SRs, particularly whether necessary information related to CHM was adequately reported. Methods The Cochrane Database was systematically searched for all SRs of CHM that were published up to 31 December 2017. The primary analysis was to assess their reporting quality based on 27-item of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) and 9-item of CHM-related information designed according to TCM theory. Descriptive statistics were additionally used to analyze their baseline characteristics. Results A total of 109 Cochrane SRs of CHM were identified from 1999 to 2017. For 27-item of PRISMA, 26 had the reporting compliances higher than 50%, of which 11 were fully reporting (100%). However, for CHM-related information, 65 (59.6%) SRs did not report the specific name of the CHM in the title, 42 (38.5%) lacked TCM-related rationales in the introduction, 62 (56.9%) did not include CHM-related characteristics in the additional analyses, and 77 (70.6%) did not analyze CHM results in terms of TCM-related theories in the discussion. Of 97 SRs that included clinical trials, 38 (39.2%) did not provide the details of composition and dosage of CHMs, 85 (87.6%) did not report the CHM sources, 13 (13.4%) did not provide the dosage form, 95 (97.9%) lacked CHM quality control information, and 57 (58.8%) did not describe details of the controls. For 62 (72.9%) of 85 SRs that included meta-analysis, it was impossible to assess whether meta-analysis had been properly conducted due to inadequate reporting of CHM interventions. Conclusion Although the Cochrane SRs of CHM showed reporting compliance with PRISMA checklist, their reporting quality needs improvement, especially about full reporting of CHM interventions and of TCM-related rationales. Reporting guideline of “PRISMA extension for CHM interventions” should be developed thus to improve their quality.


24
or/18-23 25 17 or 24 S2: Reporting quality assessment rules of included CDSRs. It should contain a word as "systematic review", "meta-analysis", or both. But in this study, item 1 was not assessed in the CDSRs.
Abstract 2 Structured summary There was a structured summary including, as applicable, background, objectives, method (data sources, study eligibility criteria, participants, interventions, study appraisal and synthesis methods), results, conclusions and implications of key findings.
Introduction 3 Rationale Described the rationale for the review in the context of what is already known. For the CDSRs, this content could be found in the section of background including "Description of the condition and intervention; How the intervention might work; and Why it is important to do this review".

4
Objective Provided the information of questions being addressed with reference to PICOS (participants, interventions, comparisons, outcome and study design).

Protocol and registration
The information of a review protocol could be found, such as web address or its information could be mentioned in the review (e.g., History; Differences between protocol and review), it is considered "yes".
6 Eligibility criteria Provided the study characteristics (such as PICOS) used as criteria for eligibility, it is considered "yes".
7 Information sources Described all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched. It is considered "yes" if all this detailed information has been reported clearly.

Search
If the review did not include any Chinese database, it is considered "yes" when it has reported electronic search strategy used for at least one database. But if any Chinese database was included in the review, it could be at least reported the search strategy of one Chinese database, it is considered "yes".

9
Study selection Reported the process for selecting studies (e.g., "Two reviewers independent selecting studies based on eligibility criteria).
10 Data collection process Described method of data extraction from reports (such as piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.
11 Data items Listed all variables for which data were sought (such as PICOS, funding sources) and any assumptions and simplifications made.

Risk of bias in individual studies
Described the assessment tool and methods used for assessing risk of bias or methodology quality of individual studies, and how this information is to be used in any data synthesis. Provided the information about publication bias or selective reporting within studies.
Although publication bias wasn't performed, its reason could be found, it is considered "yes". consider their relevance to key groups.
25 Limitations Discussed limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias). For example, the description of limitations could be found in the section of "Potential biases in the review process" in the CDSRs.

Conclusions
Provided a general interpretation of the results in the context of other evidence, and implications for future research. For example, it is can be found in the section of "Authors' conclusions" in the CDSRs.

Funding 27 Funding
Described sources of funding for the review and other support (such as supply of data). It is better to report the role of funders for the systematic review. This item could be found in the section of "Source of support" in the CDSRs. In the CDSRs, the section of "Background" included the introduction and rationale of the "condition", "intervention", and "how the intervention might work", etc. It is considered as "yes" if this part of an SR had reported TCM theory related to the condition and/or intervention, no matter more or less.

3.Information source
Was the search strategy reported , it is impossible to assess whether the meta-analyses in the SRs were properly conducted or not.
Specifically, it is considered as "yes" if an SR had completely reported the above 6) and 7) item specifics. If an SR had not reported enough information to ensure that no heterogeneity was existed in the CHM interventions, but it had conducted the meta-analyses. This is considered as "unclear" that represents "impossible to assess".

9.Summary of evidence and limitations
Was the discussion reporting the interpretation from the perspective of TCM theory?
For CDSRs, the "Discussion" includes some sections of "Summary of main results", "Overall completeness and applicability of evidence", "Quality of the evidence", and "Potential biases in the review process", etc. Anywhere had reported discussion contents related to TCM theories has been considered as "yes". No matter more or less of its words.