Skip to main content
Download PDF

The reporting quality of meta-epidemiological studies needs substantial improvement: a research on research study

Systematic Reviews volume 13, Article number: 244 (2024) Cite this article

Abstract

Background

Meta-epidemiological research plays a vital role in providing empirical evidence needed to develop methodological manuals and tools, but the reporting quality has not been comprehensively assessed, and the influence of reporting guidelines remains unclear. The current study aims to evaluate the reporting quality of meta-epidemiological studies, assess the impact of reporting guidelines, and identify factors influencing reporting quality.

Methods

We searched PubMed and Embase for meta-epidemiological studies. The reporting quality of these studies was assessed for adherence to established reporting guidelines. Two researchers independently screened the studies and assessed the quality of the included studies. Time-series segmented linear regression was used to evaluate changes in reporting quality over time, while beta-regression analysis was performed to identify factors significantly associated with reporting quality.

Results

We initially identified 1720 articles, of which 125 meta-epidemiological studies met the inclusion criteria. Of these, 65 (52%) had low reporting quality, 60 (48%) had moderate quality, and none achieved high quality. Of the 24 items derived from established reporting guidelines, 4 had poor adherence, 13 had moderate adherence, and 7 had high adherences. High journal impact factor (≥ 10) (OR = 1.42, 95% CI: 1.13, 1.80; P = 0.003) and protocol registration (OR = 1.70, 95% CI: 1.30, 2.22; P < 0.001) were significantly associated with better reporting quality. The publication of the reporting guideline did not significantly increase the mean reporting quality score (− 0.53, 95% CI: − 3.37, 2.31; P = 0.67) or the trend (− 0.38, 95% CI: − 1.02, 0.26; P = 0.20).

Conclusions

Our analysis showed suboptimal reporting quality in meta-epidemiological studies, with no improvement post-2017 guidelines. This potential shortcoming could hinder stakeholders’ ability to draw reliable conclusions from these studies. While preregistration could reduce reporting bias, its adoption remains low. Registration platforms could consider creating tailored types for meta-epidemiological research, and journals need to adopt more proactive measures to enforce reporting standards.

Peer Review reports

Introduction

When clinical studies or meta-analyses are flawed, their results may be overestimated or underestimated, potentially leading to misguided decisions in clinical practice and healthcare policy [1]. Meta-epidemiological studies are a fairly new type of study that can serve as methodological investigations to assess the associations between different research characteristics and treatment effect estimates [2], allowing for the examination and adjustment of potential biases. Consequently, meta-epidemiological research is crucial in enhancing the design, implementation, evaluation, and application of clinical studies. Furthermore, it plays a vital role in providing empirical evidence needed to develop methodological manuals and tools, such as the Cochrane risk-of-bias tool, Grading of Recommendations Assessment, Development, and Evaluations (GRADE) guidelines, and guidance on non-randomized studies of interventions [3,4,5,6].

Meta-epidemiological studies evaluating characteristics like blinding, sample size, various design types, and early stopping have shown significant inconsistencies in their results [7]. Analyzing and evaluating the causes of these inconsistencies require that studies be reported clearly, transparently, and comprehensively. Meta-epidemiological studies supporting the development of methodological guidelines can significantly impact the design and conduct of clinical research if they suffer from reporting biases. Additionally, poorly reported meta-epidemiological studies with low transparency can undermine readers’ confidence in the results, hinder the translation and utilization of research findings, and ultimately lead to research waste.

In response to these limitations, guidelines for reporting meta-epidemiological methodology research were published in 2017, with the aim of improving the transparency and clarity [8]. The reporting guidelines have been incorporated into the Enhancing the QUAlity and Transparency of health Research (EQUATOR) Network [9, 10] and are specifically designed for meta-epidemiological research. While there have been efforts to evaluate the impact of reporting guidelines, their influence on the reporting quality of meta-epidemiological studies has not been fully explored. Considering the increasing prevalence of meta-epidemiological studies, which are inherently observational in nature and possess unique risks of bias, the importance of appropriate reporting becomes paramount. The objective of this study was to evaluate the reporting quality of meta-epidemiological research, examine the impact of these reporting guidelines, and investigate potential associations between the reporting quality and other factors to provide practical recommendations.

Methods

Inclusion and exclusion criteria

We incorporated published meta-epidemiological studies. We adopt the definition of meta-epidemiological studies as proposed by Sterne et al., focusing on investigations that assess the impact of trial characteristics on treatment effect estimates. This definition is widely recognized and has been influential in shaping the field. Although the term “meta-epidemiology” can encompass a broader range of studies where the unit of analysis is a study rather than individual patients, we have chosen to apply a more precise definition to ensure clarity and relevance to our research objectives. There were no restrictions concerning the interventions or disciplinary domains examined in the meta-epidemiological studies. Exclusions encompassed methodological guidelines pertaining to meta-epidemiological studies, study protocols, conference abstracts, reviews, editorial comments, and letters.

Search strategy

We searched MEDLINE (PubMed) and Embase (OVID) for published meta-epidemiological studies from their inception to June 1, 2022, without language restriction. The initial literature search was performed on November 23, 2020, and an updated search was conducted on June 1, 2022. The search strategy was developed based on the study by Dechartres et al. [7], and additional information regarding the search methods can be found in the Additional file 1.

Screening

Two independent investigators conducted a thorough evaluation of the titles and abstracts of potential studies to determine their eligibility. Full-text articles of potentially relevant meta-epidemiological studies underwent a detailed assessment for final inclusion. In the case of any disagreements, a third reviewer was consulted to reach a consensus. All authors involved in this process have accumulated over 5 years of experience in conducting meta-epidemiological studies. We used EndNote for the identification and management of references, including the identification and removal of duplicate records, and no automation tools were applied.

Data extraction

Two researchers independently collected pertinent characteristics of the included meta-epidemiological studies. These characteristics encompassed the name of the first author, the country of the corresponding author, the number of authors, the year of publication, the journal, the involvement of methodologists, protocol details, funding sources, conflict of interest declarations, and the types of diseases and interventions examined. The participation of a methodologist was determined based on reported affiliations and acknowledgements provided in the manuscripts. Affiliations were assessed to identify authors from methodological departments, such as evidence-based medicine, clinical epidemiology, or statistics. Additionally, acknowledgements were reviewed for listed roles such as statistical analysis, study design, or methodological support. In cases where uncertainty arose regarding specific articles, the data extraction was reviewed and resolved by a third researcher.

Assessment of reporting quality

The assessment of reporting quality in the included meta-epidemiological studies was conducted in accordance with the guidelines for reporting meta-epidemiological methodology research [8]. Each item outlined in the reporting guidelines was assigned a binary score, whereby 1 point was given for a “yes” response indicating complete reporting, and 0 point was assigned for a “no” response indicating incomplete reporting or the absence of the item. Two evaluators independently scored each of the included studies. Prior to the formal assessment, a pilot testing phase was performed utilizing a set of 15 sample articles. Subsequently, the evaluators underwent training facilitated by an experienced researcher, resulting in an achieved kappa statistic of 0.70. Consultation with the principal investigator was sought to address any unresolved issues or ambiguities.

Statistical analysis

The adherence to individual reporting guideline items was assessed by calculating the proportion of meta-epidemiological studies that adhered to a given item. These proportions were then classified into three categories: high adherence (> 80%), moderate adherence (between 30 and 80%), and poor adherence (< 30%) [11].

To evaluate the reporting quality of the included meta-epidemiological studies, a scoring system was employed. Each of the 24 reporting guideline items was assigned a score ranging from 0 to 1, and these scores were summed to obtain a total score, referred to as the reporting quality score. The reporting quality score for each meta-epidemiological study ranged from 0 to 24 points, with a higher score indicating better reporting quality. The reporting quality scores were further classified into three categories: high reporting quality (> 20), moderate reporting quality (between 15 and 20), and low reporting quality (≤ 15).

We presented a graphical representation of the mean reporting quality score over a period from 2012 to 2022. We employed an interrupted time-series analysis model to evaluate changes before and after the publication of the reporting guideline for meta-epidemiological studies in 2017 [12]. A time-series segmented linear regression with Newey-West standard errors was employed to account for potential heteroscedasticity [13]. The Breusch-Godfrey test was conducted to check for autocorrelation, and if the result was significant, the model was adjusted for autocorrelated errors.

We conducted a multivariable analysis using a beta-regression model to investigate the potential factors that influence the reporting quality of meta-epidemiological studies [14]. The dependent variable in this analysis was the proportion of reporting guidelines items adhered to per meta-epidemiological study. The independent variables included impact factor, number of authors, authors from ≥ 2 countries, inclusion of methodologists, authors with conflict of interest, and protocol registration. These factors were selected based on previous literature [15,16,17,18,19].

For data analysis, we utilized Stata 15.0 and R version 4.3.0. All statistical tests were conducted at a significance level of 0.05. We conducted a post hoc power analysis to assess the adequacy of our sample size. The analysis demonstrated that our study had a power of 0.969, indicating a high probability that our sample size was sufficient to detect the observed effects.

Results

Search results

We initially retrieved a total of 1720 records through our literature search, including 803 from MEDLINE and 917 from Embase. After removing duplicates and conducting title/abstract screening, 521 publications remained eligible for full-text screening. After excluding 396 studies based on full-text review (Additional file 2), we included 125 meta-epidemiological studies in our analysis (Fig. 1 and Additional file 2).

Fig. 1
figure 1

The flow chart of literature selection

Full size image

Study characteristics

These included 125 studies were published between 1995 and 2022, with a notable increase in the number of publications over the last 5 years, as evidenced by 68 (54.4%) studies published after 2017 (Fig. 2). Approximately, one-third of the studies (39, 31.2%) were published in high-impact factor journals (impact factor ≥ 10). The majority of the studies had authors from multiple countries (82, 65.6%) and involved the participation of methodologists (89, 71.2%). Conflict of interest among study authors was rare, with only 11 (8.8%) studies reporting such situations, while 99 (79.2%) studies did not register their study protocols. Most of the included studies focused on various medical conditions (69, 55.2%) and types of interventions (69, 55.2%) rather than specific diseases or interventions. Corresponding authors were predominantly affiliated with European countries (74, 59.2%), followed by America (20, 16.0%) and Canada (14, 11.2%). Table 1 provides detailed information on the included meta-epidemiological studies.

Fig. 2
figure 2

Number of publications per year (up to June 1, 2022)

Full size image
Table 1 Characteristics of included meta-epidemiological studies
Full size table

Adherence of the meta-epidemiological studies to individual reporting guidelines items

Figure 3 and Additional file 3 present the level of adherence of the included meta-epidemiological studies to each of the reporting guidelines items. Of those 24 items, 7 were highly adhered by the included meta-epidemiological studies (met by 80% or more of the meta-epidemiological studies). These included rationale (100%, 95% CI: 100%, 100%), eligibility criteria (99%, 95% CI: 98%, 100%), abstract structured summary (97%, 95% CI: 94%, 100%), funding (94%, 95% CI: 89%, 98%), limitations (88%, 95% CI: 82%, 94%), information sources (86%, 95% CI: 79%, 92%), and summary of evidence (86%, 95% CI: 79%, 92%). The other four items were poorly adhered (met by less than 30% of the meta-epidemiological studies), including risk of bias within studies (22%, 95% CI: 14%, 30%), conclusions (13%, 95% CI: 7%, 19%), study selection in results (10%, 95% CI: 5%, 16%), and objectives (10%, 95% CI: 4%, 15%). The remaining 13 items, which were moderately adhered, were risk of bias in individual studies (73%, 95% CI: 65%, 82%), additional analysis (68%, 95% CI: 60%, 76%), data items (69%, 95% CI:61%, 77%), search (69%, 95% CI:61%, 77%), title (69%, 95% CI: 61%, 77%), synthesis of results (70%, 95% CI: 62%, 79%), results of individual studies (63%, 95% CI: 55%, 72%), study characteristics (65%, 95% CI: 56%, 73%), summary measures (66%, 95% CI: 57%, 74%), data collection process (50%, 95% CI: 41%, 59%), synthesis of results (50%, 95% CI: 41%, 59%), study selection in methods (48%, 95% CI: 39%, 57%), and protocol (37%, 95% CI: 28%, 45%).

Fig. 3
figure 3

Adherence of the meta-epidemiological studies to the reporting guidelines

Full size image

Reporting quality score of the meta-epidemiological studies

As depicted in Fig. 4, the reporting quality score of the meta-epidemiological studies exhibits a distribution that spans from 4 to 20. The median score stands at 15, with the first and third quartiles being 13 and 18, respectively. Out of the 125 studies scrutinized, 65 (52%) demonstrated a low reporting quality (≤ 15 points), and 60 (48%) demonstrated a moderate reporting quality (15 ~ 20 points). However, none of the studies achieved a high reporting quality (> 20 points).

Fig. 4
figure 4

The percentage of adequately reported individual items based on the guidelines for reporting meta-epidemiological methodology research

Full size image

Change in the mean reporting quality score per meta-epidemiological study

Figure 5 and Table 2 detail the change in the mean reporting quality score per meta-epidemiological study before and after the introduction of the introduction of the reporting guideline for meta-epidemiological studies published in 2017. The publication of the reporting guideline did not result in a significant increase in either the level of the mean reporting quality score (− 0.53, 95% CI: − 3.37, 2.31; P = 0.67) or the trend (− 0.38, 95% CI: − 1.02, 0.26; P = 0.20). Both the pre-intervention (0.37, 95% CI: − 0.33, 1.06; P = 0.25) and post-intervention (− 0.01, 95% CI: − 0.30, 0.27; P = 0.91) trends also did not show statistical significance.

Fig. 5
figure 5

Change in mean reporting quality score from 2012 to 2022, before and after the introduction of the reporting guideline for meta-epidemiological studies published in 2017

Full size image
Table 2 Estimates of parameters in segmented regression models for predicting the mean annual reporting quality score per study
Full size table

Study characteristics associated with reporting quality

The results of the beta-regression analysis revealed that an impact factor of 10 or higher (OR = 1.42, 95% CI: 1.13, 1.80; P = 0.003) and the registration of the study protocol (OR = 1.70, 95% CI: 1.30, 2.22; P < 0.001) were significantly associated with higher reporting quality in meta-epidemiological study (Fig. 6). The number of authors, international collaboration, inclusion of methodologists, and authors with conflicts of interest did not exhibit a significant relationship with reporting quality.

Fig. 6
figure 6

Multivariable regression analysis with the beta-regression model of potential factors for reporting quality of meta-epidemiological studies

Full size image

Discussion

To the best of our knowledge, our study was the first to investigate the reporting quality of meta-epidemiological studies. In this study, which encompassed nearly all published meta-epidemiological studies, we discovered that the reporting quality was generally suboptimal. This finding was consistent with assessments of other emerging study designs [20,21,22,23]. None of the reviewed studies achieved high reporting quality. Despite the introduction of guidelines in 2017, we noticed no significant improvement in reporting quality scores, a trend echoed in recent studies on the reporting of real-world evidence and prognostic prediction models [24, 25]. This indicated a considerable discrepancy between the introduction of reporting guidelines and their practical application in improving reporting quality. It underscored the urgent need for improved reporting practice to enable readers to make informed judgments regarding potential biases and the reliability of the results in meta-epidemiological studies. Moreover, factors like the impact factor of journals and the registration of study protocols correlated with better reporting quality, a finding supported by Xu et al. [26]. This suggested avenues for improving reporting practices.

On reporting of meta-epidemiological studies

The observed results may stem from several factors. First, the lack of significant improvements after the introduction of reporting guidelines indicated that awareness of these guidelines might be insufficient, with journals and authors possibly not prioritizing them. For instance, some meta-epidemiological studies in recent years still report based on the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement. Secondly, although the involvement of methodologists typically fosters better reporting and methodological quality, our analysis shows that nearly 90% of the included studies had methodologist participation yet failed to achieve the expected improvements in reporting quality. This suggests that the issue may lie in the acceptance and proper understanding of the reporting guidelines. Thirdly, the association between higher reporting quality and both the journal impact factor and study registration indicated that journals and registration platforms could play a crucial role in enhancing reporting quality.

This investigation further revealed variability in adherence to specific items of reporting guidelines. Items such as the rationale, structured summary, eligibility criteria, funding, and limitations exhibited high adherence. This adherence was largely due to their similarity with the items delineated within the PRISMA statement [27]—a foundation that the meta-epidemiological reporting guidelines were adapted from. The routine application of these items in systematic reviews and meta-analyses has led to their widespread familiarity, facilitating their recognition and implementation by researchers.

However, certain items manifested low adherence. The markedly low adherence to reporting objectives in meta-epidemiological studies reflects that authors described their goals without specifying hypotheses. Hypotheses are crucial as they predetermine the analytical framework and statistical methods; without them, the interpretation of results may be compromised. For the study selection consistency, the most common issue was the authors’ failure to report the interrater agreement (kappa statistic). This omission could potentially introduce selective bias and make it difficult for readers to assess the subjectivity. The low adherence to reporting conclusions is mainly because most authors do not discuss the impact of meta-epidemiological findings on clinical practice in their conclusions. This may be due to the typically minor direct impact of such research on clinical practice, which authors might overlook. Nevertheless, reporting guidelines suggest that indirect impact on clinical practice is possible and should be mentioned. Low adherence to the protocol may be related to the early developmental stage of meta-epidemiological studies, characterized by the absence of mandatory research registration requirements and a lack of suitable registration platforms [28]. The moderate adherence to the title could be attributed to the considerable variability in definitions and terminology, particularly in earlier years. This inconsistency has made it difficult for researchers to clearly label their studies as meta-epidemiological in titles, thereby impacting the ease of indexing and retrieving these studies [29].

A combination of targeted strategies to enhance the reporting quality of meta-epidemiological studies

To improve the reporting quality in meta-epidemiological studies, we need to raise awareness of the reporting guidelines among researchers, peer reviewers, and journal editors and also shift their attitudes to promote acceptance. Since passive dissemination of these guidelines has been largely ineffective, actively ensuring compliance is crucial [30]. We propose a combination of targeted strategies, as detailed in Fig. 7.

Fig. 7
figure 7

Steps and strategies to enhance reporting quality in meta-epidemiological studies

Full size image

Compared to the widely accepted Consolidated Standards of Reporting Trials (CONSORT) and PRISMA statements published in high-impact journals, the influence of meta-epidemiological reporting guidelines remains comparatively modest. We recommend implementing targeted training courses for researchers on platforms such as the EQUATOR Network to enhance their awareness and correct understanding of reporting guidelines [10]. Furthermore, we encourage leading journals in the field of meta-epidemiology to incorporate these reporting guidelines into their instructions for authors, thereby strengthening their dissemination.

Although the majority of reporting guidelines in the EQUATOR Network were not developed using the Delphi method [31], its absence may exacerbate observed issues in the development of reporting guidelines for meta-epidemiological studies. These issues included incomplete adherence to the EQUATOR guidelines’ methodological framework [32], insufficient detail on development methodologies, and the lack of comprehensive explanatory documents. Consequently, there was a risk that items from the reporting guidelines may be misunderstood or not widely accepted. Consequently, we recommend updating the current reporting guidelines to incorporate feedback from a wide range of experts, including editors of leading meta-epidemiology journals, to ensure the updates align with the latest research and editorial standards, thereby facilitating broader dissemination and gaining support from influential journals.

Improving reporting quality requires changes in reporting practices. Firstly, journals should play a crucial role in transforming these practices. For example, journals can require authors to submit a complete checklist with their manuscripts, enabling peer reviewers to use these checklists during the review process to ensure compliance with the reporting guidelines. Alternatively, journals can deploy automated tools to screen submissions and identify any deviations from the reporting guidelines. Also, providing feedback to authors and using simple interventions like email reminders during manuscript revision can facilitate reporting improvements. Where feasible, assistant editors may make further modifications during the final stages of editing. Making adherence to reporting guidelines a prerequisite for publication can significantly promote the use of these guidelines by authors.

Secondly, we encourage researchers to strictly follow the reporting guidelines and to preregister and publish the protocols. Unregistered studies may fail to report all data due to outcome reporting bias, leading to the concealment of important information. Such bias was a major cause of distortion in existing scientific evidence, and the complementarity between registered reports and publications can significantly reduce this bias [33]. Therefore, we call on registration platforms to create new registration types specifically tailored to meta-epidemiological research needs, reducing bias and enhancing transparency. Additionally, when publishing protocol registrations, researchers should also prioritize updating protocol status, securing external funding, and maintaining high methodological quality to further increase the likelihood of publication and improve overall research integrity [34].

Lastly, to ensure continual improvement of reporting practices, we encourage stakeholders to provide direct feedback via a dedicated web platform. We propose creating a collaborative platform for regular virtual meetings and maintaining an open feedback channel, facilitating joint updates of the reporting guidelines in response to new challenges of meta-epidemiological studies. Following recommendations from the EQUATOR network [32], we also propose establishing a database to periodically review reporting quality and assess the impact of guidelines, providing data support for ongoing optimization. Given the significant costs, we recommend securing diverse funding from government, nonprofits, and private donors to ensure the platform’s sustainability and effectiveness [35, 36].

Limitations of the study

Our study has several limitations. Firstly, this study focuses on a specific set of reporting guidelines, which may not fully capture all aspects relevant to meta-epidemiological research quality. Secondly, evaluating reporting quality remains subjective, despite standardizing efforts and mandating a strict 30-min minimum assessment per study. Thirdly, the observational nature of this study means we can only report associations rather than causative relationships between adherence to reporting guidelines and improvements in reporting quality. Finally, although our search was limited to PubMed and Embase, we believe these databases provide sufficient coverage for the scope of our study.

Conclusion

Our analysis confirms that the reporting quality in meta-epidemiological studies was suboptimal, and the introduction of reporting guidelines in 2017 did not result in improvements. Consequently, stakeholders, including manual developers, clinical researchers, and methodologists, face challenges in utilizing reliable conclusions from meta-epidemiological studies to enhance the design of clinical studies. Preregistration to avoid selective reporting bias can reduce the omission of crucial information in meta-epidemiological studies or provide supplementary information. However, the proportion of included studies that were registered was very low. Registration platforms should create new registration types specifically tailored to the needs of meta-epidemiological research and encourage registration. Journals need to become much more proactive in requiring meta-epidemiologists to change their reporting practices, although improving the final quality of reports requires collaborative efforts from all stakeholders.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

GRADE:

Grading of Recommendations Assessment, Development, and Evaluations

EQUATOR:

Enhancing the QUAlity and Transparency of health Research

STROBE:

STrengthening the Reporting of OBservational studies in Epidemiology

PRISMA:

Preferred Reporting Items for Systematic reviews and Meta-Analyses

CONSORT:

Consolidated Standards of Reporting Trials

References

  1. Higgins JPT, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343: d5928.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Sterne JAC, Jüni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med. 2002;21(11):1513–24.

    Article  PubMed  Google Scholar 

  3. Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook For Systematic Reviews Of Interventions version 6.3 (updated February 2022). Cochrane. 2022;www.training.cochrane.org/handbook. Accessed 25 Jun 2024.

  4. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366: l4898.

    Article  PubMed  Google Scholar 

  5. Saldanha IJ, Adam GP, Bañez LL, Bass EB, Berliner E, Devine B, et al. Inclusion of nonrandomized studies of interventions in systematic reviews of interventions: updated guidance from the Agency for Health Care Research and Quality Effective Health Care program. J Clin Epidemiol. 2022;152:300–6.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Cuello-Garcia CA, Santesso N, Morgan RL, Verbeek J, Thayer K, Ansari MT, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol. 2022;142:200–8.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Dechartres A, Trinquart L, Faber T, Ravaud P. Empirical evaluation of which trial characteristics are associated with treatment effect estimates. J Clin Epidemiol. 2016;77:24–37.

    Article  PubMed  Google Scholar 

  8. Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evidence Based Medicine. 2017;22(4):139–42.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Groves T. Enhancing the quality and transparency of health research. BMJ. 2008;337(7661): a718.

    Article  PubMed  Google Scholar 

  10. Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010;8:24.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Xu C, Liu Y, Jia P-L, Li L, Liu T-Z, Cheng L-L, et al. The methodological quality of dose-response meta-analyses needed substantial improvement: a cross-sectional survey and proposed recommendations. J Clin Epidemiol. 2019;107:1–11.

    Article  PubMed  Google Scholar 

  12. Hartmann DP, Gottman JM, Jones RR, Gardner W, Kazdin AE, Vaught RS. Interrupted time-series analysis and its application to behavioral data. J Appl Behav Anal. 2013;13(4):543–59.

    Article  Google Scholar 

  13. Zhang F, Wagner AK, Soumerai SB, Ross-Degnan D. Methods for estimating confidence intervals in interrupted time series analyses of health interventions. J Clin Epidemiol. 2009;62(2):143–8.

    Article  PubMed  Google Scholar 

  14. Ferrari S, Cribari-Neto F. Beta regression for modelling rates and proportions. J Appl Stat. 2004;31(7):799–815.

    Article  Google Scholar 

  15. Gao Y, Cai Y, Yang K, Liu M, Shi S, Chen J, et al. Methodological and reporting quality in non-Cochrane systematic review updates could be improved: a comparative study. J Clin Epidemiol. 2020;119:36–46.

    Article  PubMed  Google Scholar 

  16. Mei F, Chen F, Hu K, Gao Q, Zhao L, Shang Y, et al. Registration and reporting quality of systematic reviews on surgical intervention: a meta-epidemiological study. J Surg Res. 2022;277:200–10.

    Article  PubMed  Google Scholar 

  17. Mehrazmay A, Karambakhsh A, Salesi M. Reporting quality assessment of randomized controlled trials published in nephrology urology monthly journal. Nephro-urology monthly. 2015;7(4): e28752.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Chen YP, Chen L, Li WF, Lee AWM, Vermorken JB, Wee J, et al. Reporting quality of randomized, controlled trials evaluating combined chemoradiotherapy in nasopharyngeal carcinoma. Int J Radiat Oncol Biol Phys. 2017;98(1):170–6.

    Article  PubMed  Google Scholar 

  19. Dotto L, Dos Santos MBF, Sarkis-Onofre R. Reporting quality of scoping reviews in dental public health. BMC Med Res Methodol. 2023;23(1):53.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Woolf B, Di Cara N, Moreno-Stokoe C, Skrivankova V, Drax K, Higgins JPT, et al. Investigating the transparency of reporting in two-sample summary data Mendelian randomization studies using the MR-Base platform. Int J Epidemiol. 2022;51(6):1943–56.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Villacampa G, Patel D, Zheng H, McAleese J, Rekowski J, Solovyeva O, et al. Assessing the reporting quality of early phase dose-finding trial protocols: a methodological review. eClinicalMedicine. 2023;60:102020.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Xue X, Tang X, Liu S, Yu T, Chen Z, Chen N, et al. A scoping review on the methodological and reporting quality of scoping reviews in China. BMC Med Res Methodol. 2024;24(1):45.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Andaur Navarro CL, Damen JAA, Takada T, Nijman SWJ, Dhiman P, Ma J, et al. Completeness of reporting of clinical prediction models developed using supervised machine learning: a systematic review. BMC Med Res Methodol. 2022;22(1):12.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Zhao R, Zhang W, Zhang Z, He C, Xu R, Tang X, et al. Evaluation of reporting quality of cohort studies using real-world data based on RECORD: systematic review. BMC Med Res Methodol. 2023;23(1):152.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Liu C, Qi Y, Liu X, Chen M, Xiong Y, Huang S, et al. The reporting of prognostic prediction models for obstetric care was poor: a cross-sectional survey of 10-year publications. BMC Med Res Methodol. 2023;23(1):9.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Xu C, Cheng L-L, Liu Y, Jia P-L, Gao M-Y, Zhang C. Protocol registration or development may benefit the design, conduct and reporting of dose-response meta-analysis: empirical evidence from a literature survey. BMC Med Res Methodol. 2019;19(1):78.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement. BMJ. 2009;339: b2535.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Papageorgiou SN. Overview provides insights on the current status and future of meta-epidemiology. J Clin Epidemiol. 2016;77:11–2.

    Article  PubMed  Google Scholar 

  29. Puljak L, Makaric ZL, Buljan I, Pieper D. What is a meta-epidemiological study? Analysis of published literature indicated heterogeneous study designs and definitions. Journal of Comparative Effectiveness Research. 2020;9(7):497–508.

    Article  PubMed  Google Scholar 

  30. Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ. 2012;344: e4178.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Banno M, Tsujimoto Y, Kataoka Y. The majority of reporting guidelines are not developed with the Delphi method: a systematic review of reporting guidelines. J Clin Epidemiol. 2020;124:50–7.

    Article  PubMed  Google Scholar 

  32. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2): e1000217.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Wieseler B, Kerekes MF, Vervoelgyi V, McGauran N, Kaiser T. Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications. BMJ. 2012;344: d8141.

    Article  PubMed  Google Scholar 

  34. Minh LHN, Le HH, Tawfik GM, Makram OM, Tieu T, Tai LLT, et al. Factors associated with successful publication for systematic review protocol registration: an analysis of 397 registered protocols. Syst Rev. 2023;12(1):93.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Simera I, Altman DG, Moher D, Schulz KF, Hoey J. Guidelines for reporting health research: the EQUATOR Network’s survey of guideline authors. PLoS Med. 2008;5(6): e139.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Howick J, Webster R, Knottnerus JA, Moher D. Do overly complex reporting guidelines remove the focus from good clinical trials? BMJ. 2021;374: n1793.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable

Funding

This work was supported by the National Natural Science Foundation of China (grant numbers 72074161 and 81873197).

Author information

Authors and Affiliations

  1. General Practice Medical Centre, West China Hospital, Sichuan University, Chengdu, People’s Republic of China

    Youlin Long

  2. Innovation Institute for Integration of Medicine and Engineering, West China Hospital, Sichuan University, Chengdu, People’s Republic of China

    Youlin Long, Xinyao Wang, Qiong Guo, Na Zhang, Ya Deng, Ruixian Tang, Zhengchi Li & Liang Du

  3. Chinese Evidence-Based Medicine Centre, West China Hospital, Sichuan University, Chengdu, 610041, People’s Republic of China

    Youlin Long, Xinyao Wang, Na Zhang, Zhengchi Li & Liang Du

  4. School of Public Health, Sichuan University, Chengdu, People’s Republic of China

    Yurong Zheng

  5. West China Medical Publishers, West China Hospital, Sichuan University, Chengdu, People’s Republic of China

    Qiong Guo

  6. Center for Education of Medical Humanities, West China Hospital, Sichuan University, Chengdu, People’s Republic of China

    Zhengchi Li

Authors
  1. Youlin Long
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yurong Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinyao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Na Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ya Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ruixian Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Zhengchi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Liang Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YL, ZL, and LD contributed to the study conception and design. YL, YZ, XW, QG, and YD acquired the data. YL, NZ, and RT conducted the statistical analysis. YL and LD drafted the manuscript. ZL and LD had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. ZL and LD provided supervision. ZL and LD is the guarantor. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Zhengchi Li or Liang Du.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Long, Y., Zheng, Y., Wang, X. et al. The reporting quality of meta-epidemiological studies needs substantial improvement: a research on research study. Syst Rev 13, 244 (2024). https://doi.org/10.1186/s13643-024-02661-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-024-02661-7

Keywords

Systematic Reviews

ISSN: 2046-4053