Prognostic factors for tumor recurrence in patients with clinical stage I seminoma undergoing surveillance—protocol for a systematic review
- Frank Kunath1, 5Email author,
- Annabel Spek2, 5,
- Katrin Jensen3,
- Friedemann Zengerling†4, 5 and
- Stefanie Schmidt†5
© Kunath et al. 2015
Received: 6 August 2015
Accepted: 7 December 2015
Published: 18 December 2015
Testicular cancer is primarily treated with the surgical removal of the affected testis. About 50 % of testicular cancer patients present with a stage I seminoma. If no chemo- or radiotherapy as adjuvant treatment is initiated after orchiectomy, 15–20 % of these patients will develop metastases. Although adjuvant treatment is effective in reducing the relapse risk, there is rising concern about overtreatment of these patients. Prognostic factors at primary diagnosis might have the potential to identify patients at higher risk of tumor relapse, allowing to guide individual therapy and to avoid overtreatment. Therefore, we aim to synthesize the available evidence on tumor or patient characteristics as possible prognostic factors for cancer recurrence in patients with clinical stage I seminoma.
We will conduct a broad systematic review to analyze what prognostic factors predict cancer recurrence in patients with a first time diagnosis of clinical stage I seminoma, who received no adjuvant chemo- or radiotherapy after orchiectomy. The literature search will comprise MEDLINE, Web of Science, the Cochrane Central Register of Controlled Trials (CENTRAL), and the conference proceedings of the American Society of Clinical Oncology (ASCO), American Urologic Association (AUA), and European Urologic Association (EAU) Annual Meetings. Prospective and retrospective longitudinal studies reporting on prognostic factors for cancer recurrence will be considered. We will consider the wealth of any candidate clinical or pathological prognostic factor reported in the literature. Our outcome of interest will be tumor recurrence at a minimum of 2 years follow-up. Study screening, data extraction, and quality assessment will be done by two reviewers independently. Hazard ratios will be used to measure the relationship between the potential prognostic factor and tumor recurrence. Meta-analyses will be conducted with sufficiently homogeneous studies and separately with respect to study design, by using the random-effects generic inverse variance model.
Limitations and strengths will be discussed in our review, and the results will be put into context with other studies in this field. Our results will help to guide evidence-based decision-making on patients with clinical stage I seminoma, allowing a better adjustment of therapies with regard to the individual patient’s risk. Our findings will furthermore help to formulate recommendations for future research.
Systematic review registration
KeywordsTesticular cancer Seminoma Relapse Rete testis invasion Tumor diameter Adjuvant therapy Surveillance Systematic review Prognostic factor
Testicular cancer is primarily treated with inguinal orchiectomy, that means the surgical removal of the affected testis. Pathological examination reveals histology of a seminoma in about 60 % of the cases, with an increasing trend . About 80 % of the diagnosed seminomas are classified as clinical stage I. This stage is characterized by the absence of metastases on radiologic examination and by normal or normalized testis tumor markers after orchiectomy. Data from large case series show that despite normal radiologic and laboratory examinations on primary diagnosis, about 15–20 % of stage I seminoma patients will develop a tumor relapse in their further live . Therefore, early additional (adjuvant) therapies, namely adjuvant chemotherapy and adjuvant irradiation of the posterior abdominal wall, are used for stage I patients because they are able to lower the relapse risk to percentages <5 % [3, 4]. Another option for patients with stage I seminoma is a surveillance strategy, which consists of a close follow-up of affected patients rather than administering any active medical intervention. Hence, surveillance, adjuvant radiation and adjuvant chemotherapy are the three therapeutic options for patients with clinical stage I seminoma.
Recent data suggest that radio- and chemotherapy harbor relevant long-term side effects, which often have been neglected in the past [5, 6]. These long-term data on chemo- or radiotherapy imply a more distinguished point of view on adjuvant therapy regimens. The ideal approach for stage I seminoma patients is to minimize overtreatment without exposing the patient to an unnecessary high risk of tumor relapse. In reality, this is difficult to achieve, but prognostic factors at primary diagnosis might have the potential to identify patients at higher risk of tumor relapse. In clinical practice, they can help to guide individual treatment and lead to reduced rates of overtreatment.
In 2002, Warde et al. published the results of a retrospective study investigating the prognostic value of histopathological findings in the primary tumor specimen for relapse. They found a tumor diameter greater than 4 cm and infiltration of the rete testis being significantly associated with tumor relapse . The same working group was unable to validate these two risk factors in a prospective study . Ensuing prospective studies led to conflicting results regarding the value of tumor diameter and rete testis infiltration [9, 10]. Both candidate prognostic factors have been used by some physicians in clinical decision making for or against adjuvant therapy, especially in favor of surveillance if both risk factors are absent. In contrast to patients with stage I non-seminoma -where pT2 stage is an accepted risk factor-, the guidelines of the European Association of Urology (EAU) and the European Society of Medical Oncology (ESMO) do not clearly recommend these prognostic factors for the choice of treatment for stage I seminoma patients [11, 12]. There were further efforts -sometimes only in retrospective series or on a small number of patients- to identify other candidate prognostic factors that are better suited to facilitate therapy guidance. A number of molecular examinations on primary tumor tissue or patient peripheral blood for different markers have been done, but none of them has entered the routine of clinical decision-making or even the therapy guidelines.
Why it is important to do this review
The continuous discussion on different candidate prognostic factors, of which some -despite lacking evidence- have partly entered in clinical practice, leads to relevant doubts and uncertainties in seminoma patients and the doctors engaged in their treatment. Therefore, it seems important to systematically review, evaluate and summarize the existing information on prognostic factors in stage I seminoma to guide clinical decision making on further treatment strategies and to inform and facilitate future intervention research.
To synthesize the available evidence on the association between primary tumor as well as clinical patient characteristics as prognostic factors and cancer recurrence in patients with clinical stage I seminoma, who received no prior adjuvant therapy.
We will conduct a broad systematic review (as opposed to a focused review that investigates evidence on only one prognostic factor) to analyze what are the candidate factors that predict cancer recurrence in clinical stage I seminoma patients.
The conduction of systematic reviews of prognostic factors is challenged by the selective reporting of prognostic factor studies , eventually leading to false-positive results, as well as by variations in methodology and reporting of findings among primary studies . To deal with these challenges, we will follow the work done by the Prognostic Research Strategy (PROGRESS) group that developed methods for prognosis research studies . Furthermore, we will follow the reporting guidelines of Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) for systematic reviews  and Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK) for (tumor marker) prognostic studies .
Types of studies
We will include prospective and retrospective longitudinal observational studies investigating the prognosis of patients with clinical stage I seminoma. We will also include intervention studies, if they include a surveillance group and if results are reported separately for this group. Studies are eligible for inclusion if they report on patients diagnosed with stage I tumor and provide data with a minimum follow-up (median or mean) of 2 years. A minimum follow-up of 2 years is required to exclude relevant uncertainties of possible diagnostic factors, as about 30 % of seminoma relapses are diagnosed in the second year of follow-up and a 2-year follow-up covers at least about 75 % of all relapses . Exploratory studies as well as confirmatory studies will be considered , if they provide data on the association of clinical or pathohistological factors with tumor recurrence. Exploratory and confirmatory studies will be treated differently in the analyses as they provide different levels of evidence. We will also consider studies published as abstracts; their limited significance will be considered in the quality assessment process. Study inclusion will not be restricted by publication status. We will exclude reviews, case reports, editorials and comments.
Types of participants
We will include patients with clinical stage I seminoma, defined as the absence of metastasis shown with radiological imaging and normal or normalized values for the tumor marker Beta-human chorionic gonadotropin (betaHCG). Eligible are patients of all ages, ethnicity and comorbidity with a first time diagnosis of seminoma stage I, who received no prior adjuvant treatment after orchiectomy. We will exclude patients who received any adjuvant therapy (e.g., chemo- or radiotherapy).
Types of prognostic factors
We will not focus at a specific prognostic factor but will consider the wealth of any candidate clinical or pathological prognostic factor reported in the literature. So far, especially tumor size (>4 cm) and tumor infiltration into rete testis have been reported to have a possible prognostic value in predicting tumor recurrence in seminoma patients at clinical stage I . If we identify any other candidate prognostic factor during our review process, we will also include it in our analyses.
Our outcome of interest will be tumor recurrence during the observed follow-up period, as the risk for tumor recurrence is the most important factor for decisions on adjuvant therapy regimens. Tumor recurrence will be assessed as reported by the authors, regardless of the mode of detection (radiologic finding, histologic specimen, tumor marker elevation) at the described time points.
Search methods for identification of studies
We will conduct a systematic review of the literature in different electronic biomedical databases. Our search will be complemented by additional handsearching of meeting communications of the major conferences of this topic. The searches were conducted in January 2015. A final update search will be done before submitting the final review draft for publication.
Specific search strategies will be conducted for each of the following databases: MEDLINE (from 1946 onwards), Web of Science (from 1946 onwards), and the Cochrane Central Register of Controlled Trials (CENTRAL) (from 1995 onwards). The search strategies are shown in Appendix 1, Appendix 2, and Appendix 3. Search terms will combine keywords and free text. No date or language restrictions will be applied.
Searching other resources
We will handsearch the conference proceedings of American Society of Clinical Oncology (ASCO), the Annual Meeting of the European Association of Urology (EAU), and the American Urologic Association (AUA) (American Urologic Association) from 2008 onwards. Additionally, we will review the reference lists of the full texts of potential relevant studies.
Data collection and analyses
Selection of studies
Articles will be selected for inclusion by title and abstract review, performed by two authors independently. The full text of the potentially relevant references will be reviewed by the same authors independently, too. We will provide a PRISMA study flow diagram with reasons for exclusion at full text screen level. Discrepancies will be resolved by discussion or by consulting a third review author.
Data extraction and management
Data extraction will be done independently by two review authors. Disagreement will be resolved through discussion or by consulting a third review author. The data extraction form is inspired by the work of Moons et al. and adapted for our purpose . The following data will be extracted: source of data (study design, study dates, sample size, number of events per candidate prognostic factor, exploratory/confirmatory approach, follow-up length), participant characteristics (setting, inclusion and exclusion criteria, recruitment method, sociodemographic characteristics, details of treatment received, clinical tumor characteristics), candidate prognostic factors (number and type of prognostic factor, definition, method and timing of measurement of candidate prognostic factors, all unadjusted (simple) and adjusted (multivariable) associations reported between the prognostic factor and the outcome, with details on any adjustment factors that are used), outcomes (type of outcome, definition, measurement and timing of outcome assessment, time of outcome occurrence), and missing data (number per patient/per candidate prognostic factor, handling of missing data). The data extraction sheet will be pilot-tested to ensure its performance.
We will classify included studies according to the authors’ objectives, study design, definition of variables, and analysis (exploratory versus confirmatory intent). This information will be considered when interpreting the strength of the available evidence.
We will not consider individual patient data as the collection of such data is not feasible. Instead, we will contact the authors of the original publication in case of unclear or missing data.
If we identify several publications of the same study, we will consider the most recent and comprehensive one in the analysis.
All references will be managed and stored in EndNote X7.
Assessment of risk of bias of included studies
Included studies will be critically appraised by two review authors independently, using the Quality in Prognosis Studies (QUIPS) tool, which is recommended by the Cochrane Prognosis Method Group for prognostic factor review questions. Discrepancies will be resolved through discussion and reaching a consensus. If necessary, a third author will be included. The QUIPS tool considers six domains of assessment: study participation, study attrition, prognostic factor measurement, outcome measurement, study confounding, and statistical analysis and reporting . Results will be presented in a risk of bias table.
Data analysis and synthesis
Measures of association
Hazard ratios (HRs) will be used to measure the relationship between the potential prognostic factor and tumor recurrence (time to event). We will extract all unadjusted and adjusted measures of association. An indirect estimation method will be used to calculate log HRs and their standard errors if they are not reported but adequate univariate analyses are available . Odds ratios and relative risks will be used to estimate HRs as necessary . For consistency, we will re-calculate associations to be in the same direction, as necessary, with associations above 1 indicating higher risk of relapse/worse prognosis. Standard errors will be calculated from confidence intervals and the individual study associations. Standard errors will be appropriately transformed to their natural logarithms if necessary .
Unit of analysis
Data retrieved from the primary studies is analyzed at the group level. We are aware that group or study level analyses can lead to biased assessments and have some limitations in revealing subset effects. But, in view of our broad approach including a set of factors investigated through a very long time span, we felt that an individual patient level analysis is not feasible for the present issue.
Dealing with missing data
We will include all studies investigating the association of clinical or pathological factors on tumor recurrence. Studies with missing data or not significant results will also be included. We will contact the corresponding author to request any important missing information or clarify questions, if necessary.
We will investigate publication bias by creating funnel plots for each meta-analysis containing at least ten studies. Publication bias will be assessed by visually examining the asymmetry of the funnel plot and by testing for asymmetry at the 10 % level, using Egger’s test for HRs and Peters’ test for odds ratios . Furthermore, we will contact experts in the field to identify unpublished studies.
Meta-analyses will only be conducted if valid data are available and data are sufficiently homogeneous for two or more studies. Data synthesis will be done separately with respect to differences in study design and study quality by excluding studies with low and/or unclear risk of bias.
In the meta-analyses, the random-effects generic inverse variance model will be used, which accounts for any between-study heterogeneity in the prognostic effect. Such heterogeneity is common in prognostic factor studies. Each meta-analysis will be summarized by the pooled estimate (the average prognostic factor effect), its 95 % confidence interval, the estimate of Tau2 (between-study variance), and a 95 % prediction interval for the prognostic effect in a single population [25, 26].
For continuous prognostic factors, meta-analyses will be conducted on the same scale. For dichotomous or categorical prognostic factors, we will group studies with similar cut points together to obtain meta-analysis results for each cut point as far as possible. To allow combination of as much data as possible, dichotomous associations will be computed from continuous and categorical measures of association from data in the study reports or provided by the study authors on request. We will explore the impact of data transformations using sensitivity analyses.
Assessment of heterogeneity
0 to 40 % might not be important
30 % to 60 % may represent moderate heterogeneity
50 % to 90 % may represent substantial heterogeneity
75 % to 100 % considerable heterogeneity
Clinical heterogeneity will be examined based on patient characteristics, clinical and pathological characteristics, and outcome measurement. Methodological heterogeneity will be examined based on study design and potential biases.
If it is not appropriate to combine the results in a meta-analysis (due to a small number of studies and/or if the interpretation of results would be difficult due to heterogeneity), they will be presented qualitatively by considering the strength and consistency of results. Therefore, the following schema will be used:
Strong evidence of effect: Consistent findings (defined as >75 % of studies showing the same direction of effect) in multiple low risk of bias studies
Moderate evidence of effect: Consistent findings in multiple high risk of bias and/or one study with low risk of bias
Limited evidence of effect: One study available
Conflicting evidence: Inconsistent findings across studies
No evidence: No association between patient expectations and the outcome of interest
The robustness of results will be tested by sensitivity analyses.
Exploratory versus confirmatory study results
Retrospective versus prospective designs
Categorized versus continuous data of the same factor
Significant associations versus non-significant associations
Adjusted versus non adjusted analyses
Follow-up time (2 years, 3 years, 5 years, 10 years)
Data synthesis will be performed using Review Manager 5.3 provided by The Cochrane Collaboration . Statistical methods not available in the Review Manager will be done using the statistical software R with the “meta” package [29, 30].
The value of prognostic markers for a tumor relapse in patients with clinically organ-confined stage I seminoma is discussed controversially in the medical literature. There is only little evidence from primary research, making decisions on adjuvant treatment for these patients difficult. Therefore, we will conduct a broad systematic review to identify prognostic factors that might be able to predict cancer recurrence in patients with clinical stage I seminoma. Current limitations of prognostic factor research is challenged by publication bias, reporting bias, poor statistical analyses, and inadequate reporting of methods and findings of primary studies. But within the increasing body of evidence, it becomes important to summarize the available information and thereby overcoming the limited validity of primary studies.
Systematic reviews of prognostic factors also provide several challenges, as they have been criticized for their limitation in providing relevant information due to missing quality appraisal of primary studies and poor adherence to standardized methodology and reporting. To overcome these challenges, we will follow the recommendations of conducting systematic reviews of prognostic studies, provided by the Cochrane Method Group of Prognosis Reviews and the PRISMA and REMARK reporting guidelines for systematic reviews and prognostic studies. Study appraisal for prognostic reviews is more difficult than for interventional reviews because there has been less research and funding in this field so far. We will use the QUIPS tool, recently described by Hayden et al. and recommended by the Cochrane Prognosis Methods Group .
Our results of the aggregation and critical appraisal of the available evidence will help to guide evidence-based treatment decision-making on patients with clinical stage I seminoma, allowing proper adjustment of therapies with regard to the individual patient’s risk. The discussion part of our study will address the limitations and strengths of this study. Our findings will be compared to results of other studies in this field. Our findings will furthermore help to formulate recommendations for future research, and clinical implications will be discussed.
American Society of Clinical Oncology
American Urologic Association
Cochrane Central Register of Controlled Trials
Deutsche Gesellschaft für Urologie
European Association of Urology
European Society of Medical Oncology
Preferred Reporting Items for Systematic Reviews and Meta-analyses
Prognostic Research Strategy
Quality in Prognosis Studies
Reporting Recommendations for Tumor Marker Prognostic Studies
We thank Prof. Dr. Bernd Wullich and the Deutsche Gesellschaft für Urologie for setting up UroEvidence, serving as scientific context for our work.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Ruf CG, Isbarn H, Wagner W, Fisch M, Matthies C, Dieckmann KP. Changes in epidemiologic features of testicular germ cell cancer: age at diagnosis and relative frequency of seminoma are constantly and significantly increasing. Urol Oncol. 2014;32(1):e1–6. doi:10.1016/j.urolonc.2012.12.002.PubMedView ArticleGoogle Scholar
- Mortensen MS, Lauritsen J, Gundgaard MG, Agerbaek M, Holm NV, Christensen IJ, et al. A nationwide cohort study of stage I seminoma patients followed on a surveillance program. Eur Urol. 2014;66(6):1172–8. doi:10.1016/j.eururo.2014.07.001.PubMedView ArticleGoogle Scholar
- Fossa SD, Horwich A, Russell JM, Roberts JT, Cullen MH, Hodson NJ, et al. Optimal planning target volume for stage I testicular seminoma: a medical research council randomized trial. Medical research council testicular tumor working group. J Clin Oncol. 1999;17(4):1146.PubMedGoogle Scholar
- Oliver RT, Mason MD, Mead GM, von der Maase H, Rustin GJ, Joffe JK, et al. Radiotherapy versus single-dose carboplatin in adjuvant treatment of stage I seminoma: a randomised trial. Lancet. 2005;366(9482):293–300. doi:10.1016/s0140-6736(05)66984-x.PubMedView ArticleGoogle Scholar
- Lewinshtein D, Gulati R, Nelson PS, Porter CR. Incidence of second malignancies after external beam radiotherapy for clinical stage I testicular seminoma. BJU Int. 2012;109(5):706–12. doi:10.1111/j.1464-410X.2011.10424.x.PubMedView ArticleGoogle Scholar
- Haugnes HS, Wethal T, Aass N, Dahl O, Klepp O, Langberg CW, et al. Cardiovascular risk factors and morbidity in long-term survivors of testicular cancer: a 20-year follow-up study. J Clin Oncol. 2010;28(30):4649–57. doi:10.1200/jco.2010.29.9362.PubMedView ArticleGoogle Scholar
- Warde P, Specht L, Horwich A, Oliver T, Panzarella T, Gospodarowicz M, et al. Prognostic factors for relapse in stage I seminoma managed by surveillance: a pooled analysis. J Clin Oncol. 2002;20(22):4448–52.PubMedView ArticleGoogle Scholar
- Chung G, Tyldesley S, Panzarella T, Kollmannsberger C, Gospodarowicz M, Warde P, et al. Prognostic factors for relapse in stage I seminoma managed with surveillance: a validation study. J Clin Oncol ASCO. 2010;28(15):4535.Google Scholar
- Aparicio J, Maroto P, del Muro XG, Guma J, Sanchez-Munoz A, Margeli M, et al. Risk-adapted treatment in clinical stage I testicular seminoma: the third Spanish Germ Cell Cancer group study. J Clin Oncol. 2011;29(35):4677–81. doi:10.1200/jco.2011.36.0503.PubMedView ArticleGoogle Scholar
- Tandstad T, Smaaland R, Solberg A, Bremnes RM, Langberg CW, Laurell A, et al. Management of seminomatous testicular cancer: a binational prospective population-based study from the Swedish norwegian testicular cancer study group. J Clin Oncol. 2011;29(6):719–25. doi:10.1200/jco.2010.30.1044.PubMedView ArticleGoogle Scholar
- Albers P, Albrecht W, Algaba F, Bokemeyer C, Cohn-Cedermark G, Fizazi K et al. EAU guidelines on testicular cancer: 2015 update http://uroweb.org/wp-content/uploads/EAU-Guidelines-Testicular-Cancer-2015-v2.pdf. Accessed 19 July 2015.
- Oldenburg J, Fossa SD, Nuver J, Heidenreich A, Schmoll HJ, Bokemeyer C, et al. Testicular seminoma and non-seminoma: ESMO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2013;24(6):vi125–32. doi:10.1093/annonc/mdt304.PubMedGoogle Scholar
- Kyzas PA, Loizou KT, Ioannidis JP. Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst. 2005;97(14):1043–55. doi:10.1093/jnci/dji184.PubMedView ArticleGoogle Scholar
- Altman DG. Systematic reviews of evaluations of prognostic variables. BMJ. 2001;323(7306):224–8.PubMed CentralPubMedView ArticleGoogle Scholar
- Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, et al. Prognosis research strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013;10(2), e1001380. doi:10.1371/journal.pmed.1001380.PubMed CentralPubMedView ArticleGoogle Scholar
- Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006–12. doi:10.1016/j.jclinepi.2009.06.005.PubMedView ArticleGoogle Scholar
- Altman DG, McShane LM, Sauerbrei W, Taube SE. Reporting recommendations for tumor marker prognostic studies (REMARK): explanation and elaboration. BMC Med. 2012;10:51. doi:10.1186/1741-7015-10-51.PubMed CentralPubMedView ArticleGoogle Scholar
- Hayden JA, Cote P, Steenstra IA, Bombardier C. Identifying phases of investigation helps planning, appraising, and applying the results of explanatory prognosis studies. J Clin Epidemiol. 2008;61(6):552–60. doi:10.1016/j.jclinepi.2007.08.005.PubMedView ArticleGoogle Scholar
- Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 2014;11(10), e1001744. doi:10.1371/journal.pmed.1001744.PubMed CentralPubMedView ArticleGoogle Scholar
- Hayden JA, van der Windt DA, Cartwright JL, Cote P, Bombardier C. Assessing bias in studies of prognostic factors. Ann Intern Med. 2013;158(4):280–6. doi:10.7326/0003-4819-158-4-201302190-00009.PubMedView ArticleGoogle Scholar
- Parmar MK, Torri V, Stewart L. Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med. 1998;17(24):2815–34.PubMedView ArticleGoogle Scholar
- Symons MJ, Moore DT. Hazard rate ratio and prospective epidemiological studies. J Clin Epidemiol. 2002;55(9):893–9.PubMedView ArticleGoogle Scholar
- Higgins JPT, Green S. Chapter 7: selecting studies and collecting data. Cochrane Handbook of Systematic Reviews of Interventions, Version 5.1.0 [updated March 2011]. Cochrane Collaboration; 2011.www.cochrane-handbook.org
- Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. PubMedView ArticleGoogle Scholar
- Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549. doi:10.1136/bmj.d549.PubMedView ArticleGoogle Scholar
- Hayden JA, Tougas ME, Riley R, Iles R, Pincus T. Individual recovery expectations and prognosis of outcomes in non-specific low back pain: prognostic factor exemplar review (Protocol). Cochrane Database Syst Rev. 2014. doi:10.1002/14651858.CD011284.Google Scholar
- Deeks JJ, Higgins JPT, Altman DG, Group obotCSM. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. Cochrane Collaboration; 2011.
- The Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager (RevMan). 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014.
- RCT. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2015.Google Scholar
- Schwarzer G. Meta: general package for meta-analysis, Version: 4.3-0. 2015.Google Scholar