- Open Access
- Open Peer Review
Short-term versus long-term psychotherapy for adult psychiatric disorders: a protocol for a systematic review with meta-analysis and trial sequential analysis
Systematic Reviewsvolume 8, Article number: 169 (2019)
Psychiatric disorders are highly prevalent and associated with great symptomatic, functional, and health economic burdens. Psychotherapy is among the recommended and used interventions for most psychiatric disorders and is becoming widely accessible in mental health systems. The effects of specific forms of psychotherapy (e.g., psychodynamic therapies, cognitive and behavioral therapies, humanistic therapies, and systemic therapies) have been assessed previously in systematic reviews, but the appropriate psychotherapy duration for psychiatric disorders has not been reviewed. The aim of this systematic review will be to synthesize the evidence of the effects of short-term compared with long-term psychotherapy for all adult psychiatric disorders.
A comprehensive search for relevant published literature will be undertaken in Cochrane Central Register of Controlled Trials (CENTRAL), Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica database (EMBASE), Latin American and Caribbean Health Sciences Literature (LILACS), PsycINFO, Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Conference Proceedings Citation Index—Science (CPCI-S), and Conference Proceedings Citation Index—Social Science & Humanities (CPCI-SSH) to identify relevant trials. We will search all databases from their inception to the present. We will include randomized clinical trials comparing a short-term and a long-term version of the same psychotherapy type for adult psychiatric disorders including attention deficit hyperactivity disorder, psychotic disorders, depressive disorders, bipolar disorders, anxiety disorders, obsessive-compulsive disorder, trauma- and stressor-related disorders, eating disorders, and personality disorders (as defined by standardized diagnostic criteria). We will rely on the trialists defining their compared interventions as short term and long term (or similar terminology). Primary outcomes will be quality of life, serious adverse events, and symptom severity. Secondary outcomes will be suicide or suicide attempts, self-harm, and level of functioning. Two review authors will independently extract data and perform risk of bias assessment using the Cochrane risk of bias tool. A meta-analysis will be performed as recommended by the Cochrane Handbook for Systematic Review of Interventions, bias will be assessed with domains, and Trial Sequential Analysis will be conducted to control random errors. Certainty of the evidence will be assessed by GRADE.
As psychotherapy is among the treatments of choice for most adult psychiatric disorders, a systematic review evaluating the benefits and harms of short-term compared with long-term psychotherapy is urgently needed. It is the hope that this review will be able to inform best practice in treatment and clinical research of these highly prevalent and burdensome disorders.
Systematic review registration
It is estimated that each year, 38.2% of the European population suffer from a psychiatric disorder . The economic burden from psychiatric disorders is excessive, not only because of high direct health care costs, but also because of indirect costs like sick days, disability, and early retirement . Psychotherapy is among the recommended and widely used interventions for most disorders . Specific types of psychotherapy have already been systematically reviewed, but the appropriate length of psychotherapy for all adult psychiatric disorders has not been reviewed previously. To present a complete overview of the evidence and to increase the statistical power, we will therefore in the present review include any adult psychiatric disorder. The major categories of adult psychiatric disorders listed in the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-V)  are the following.
Attention Deficit Hyperactivity Disorder (ADHD) is characterized by a persistent pattern of inattention and/or hyperactivity and impulsivity that significantly interferes with functioning and development . ADHD is one of the most common psychiatric disorders of childhood and adolescence, and it often persists into adulthood. The predominant characteristics of adult ADHD differ from typical ADHD characteristics in children. Symptoms of hyperactivity or impulsivity are typically less obvious in adults, whereas symptoms of inattention are more dominant . Epidemiologic studies of adult ADHD have estimated the current prevalence to be 4.4% in the USA and 3.4% internationally [4, 5]. The total economic burden of ADHD in America has been estimated to be 31.6 billion US dollars in 2010  including both direct costs, other health care costs, health care costs for family members, and work loss of patients and their relatives.
Psychotic disorders are characterized by abnormalities in one or more of the following five domains: delusions, hallucinations, disorganized thinking, disorganized or abnormal motor behavior, and negative symptoms . The estimated annual prevalence of all psychotic disorders is 2.6% . The most common psychotic disorder is schizophrenia with an estimated median lifetime prevalence of 4.0 per 1000 and a lifetime morbid risk of 7.2 per 1000 . Annual costs for the schizophrenia population have been systematically reviewed and estimated to range from 94 million to 102 billion US dollars. Indirect costs contributed to 50–85% of the total costs associated with schizophrenia .
Bipolar disorders are characterized by serious mood changes involving mood elevation (mania or hypomania) either alone or followed by major depressive episodes . Bipolar disorder subtypes include bipolar I and bipolar II. Bipolar I disorder is associated with manic episodes nearly always followed by major depressive and hypomanic episodes. Bipolar II disorder is associated with at least one hypomanic episode, at least one major depressive episode, and the absence of manic episodes. The international annual prevalence is estimated to be 0.4% for bipolar I disorder and 0.3% for bipolar II disorder . In 2009, the estimated annual direct and indirect costs of bipolar I and II disorders were 30.7 and 120.3 billion US dollars, respectively .
Depressive disorders are characterized by the presence of a sad, empty, and irritable mood often accompanied by somatic and cognitive changes resulting in significant functional impairment . The most common depressive disorder is major depressive disorder (unipolar depression) with an annual prevalence of approximately 7% both in Europe  and in the USA . The estimated annual economic burden of adults with major depressive disorder, including direct medical costs, workplace costs, and costs associated with comorbidities exceeded 200 billion US dollars  in the USA in 2010.
Anxiety disorders are characterized by excessive and counterproductive feelings of fear and anxiety often accompanied by behavioral disturbances such as pervasive avoidance behaviors . Different anxiety disorders exist, which differ from one another in the types of objects or situations that induce hyperarousal or avoidance behavior . The prevalence of anxiety disorders is estimated to be 18% in the USA  and 14% in European countries , placing them among the most prevalent psychiatric disorders worldwide. Costs associated with anxiety disorders have previously been reported to be 46.6 billion US dollars in the USA  including both direct and indirect costs.
Obsessive-compulsive disorder (OCD) is characterized by recurrent intrusive thoughts, images, or urges (obsessions) with or without repetitive mental or behavioral acts (compulsions) . OCD among adults has an annual prevalence of 1.2% and a lifetime prevalence of 2.3% [16, 17]. The annual economic burden of OCD is estimated to be 2272 euros per patient when including both direct and indirect costs .
Trauma- and stressor-related disorders are characterized by psychological distress following exposure to a traumatic or stressful event. The most common trauma disorder is post-traumatic stress disorder (PTSD) . PTSD is a prevalent and disabling disorder associated with delayed help seeking . The estimated annual prevalence of PTSD is 2% in Europe  and 4.7% in the USA , and the estimated lifetime prevalence is 3.9% across 26 countries ranging from low to high income . The total costs of PTSD per patient have been estimated to 1082 million euros including both direct and indirect costs .
Eating disorders are characterized by a persistent disturbance in eating behavior resulting in altered consumption or absorption of food that significantly impairs health and psychosocial functioning . The most common eating disorders are anorexia nervosa, bulimia nervosa, and binge-eating disorder. Lifetime prevalence of anorexia nervosa, bulimia nervosa, and binge-eating disorder are estimated to be 0.9, 1.5, and 3.5%, respectively, among women and 0.3, 0.5, and 2.0%, respectively, among men . The estimated annual prevalence of eating disorders is 0.9% in the European population . Annual costs per patient are estimated to range from 1288 to 8042 US dollars .
Personality disorders are characterized by enduring and inflexible patterns of emotional, behavioral, and interpersonal problems that deviate markedly from cultural expectations. According to DSM-V, the following nine personality disorders exist. Personality disorders onset in adolescence or early adulthood and are associated with great psychosocial distress and impairment . In a systematic review of the economic burden of personality disorders, the estimated direct and indirect costs were 11,126 euros for patients 12 months prior to seeking treatment. Direct medical costs accounted for two thirds of these costs, while the remaining costs were related to productivity losses .
Description of the interventions
Different schools of psychotherapy exist. They are often divided into the following categories: psychodynamic therapies, cognitive and behavioral therapies, humanistic therapies, and systemic therapies .
Psychodynamic (or psychoanalytical) therapies encompass the many approaches that are influenced by Freud’s psychoanalysis but have developed into different independent schools . Traditionally, psychodynamic therapies have been considered as long-term therapies, perhaps due to the notion that the uncovering of unconscious emotions and conflicts cannot be achieved with a fixed time limit . Long-term psychoanalytical psychotherapy has previously been systematically reviewed yielding different results [26, 27]. Today, different lengths of psychodynamic therapies have been developed to treat different forms of psychopathology. In addition to traditional psychoanalysis, examples of long-term psychodynamic treatments are transference-focused psychotherapy (TFP), a psychodynamic treatment rooted in object relations theory lasting up to 3 years [28, 29], and mentalization-based therapy , an 18-month psychodynamic treatment rooted in attachment theory. Both are developed specifically to treat borderline personality disorder. Further, different variations of short-term psychodynamic therapy have been developed to treat a variation of common psychiatric disorders, most notably anxiety disorders, depressive disorders, certain behavior disorders, and personality disorders . Short-term psychodynamic therapies vary in treatment duration but typically last between 12 and 24 sessions .
Cognitive and behavioral therapies (CBT) encompass many integrative approaches. Historically, behavior therapy (first wave CBT) developed from the learning theories of Pavlov  and Skinner . An integration of a cognitive component to classical behavioral theories was first established by Beck [34, 35], who developed what is now often referred to as second-wave CBT. CBT is now often delivered as a short-term treatment, typically lasting between 12 and 20 sessions, for a variation of common psychiatric disorders like depressive disorders , anxiety disorders , obsessive-compulsive disorder , personality disorders , and eating disorders . Different durations of CBT are also available for the treatment of schizophrenia [40, 41]. Today, so-called third-wave cognitive therapies have emerged, characterized by more integrative approaches to psychotherapy, incorporating techniques from Buddhist mindfulness, psychodynamic therapies, or Gestalt therapy . These include dialectical behavior therapy  and schema-focused therapy (SFT) [29, 43], which are both long-term therapies for borderline personality disorder (up to 3 years), and acceptance and commitment therapy (ACT)  and compassion-focused therapy (CFT) , which are often delivered as short-term treatments for various psychiatric disorders [46, 47].
Humanistic therapies are characterized by psychotherapy approaches derived from humanistic and existentialist philosophy. Major approaches within this orientation are person-centered therapy , Gestalt therapy , existential psychotherapy , and process-experiential/emotion-focused therapy . All humanistic therapies share the notion of empathic understanding, the promotion of in-therapy experiencing, and a belief in the uniquely human growth tendency by applying a consistent person-centered view involving concern for each patient’s individual experience and differing needs . Humanistic therapies have not been developed to treat specific types of disorders and are traditionally considered open-ended, which is also aligned with the person-centered way of thinking. However, different lengths of humanistic therapies have been studied, e.g., PE-EFT as a short-term treatment (down to 5 weeks) for depressive disorders  and as a 20-week treatment for trauma-related disorders .
Systemic therapies are characterized by a systemic approach to psychotherapy defining patients’ problems as contextually rather than individually derived. Most often, the context of interest is the partner or the family, but it can also be a broader context, such as the extended family or a classroom . Different systemic therapies exist for different types of psychopathology. Examples are family-based therapy for eating disorders , attachment-based family therapy for depressed adolescents (ABFT) , parent management training for childhood conduct disorders , psychoeducational family interventions for schizophrenia  and bipolar disorder , and systemic treatments for substance-use disorders [59, 60]. Different lengths of systemic therapies exist. However, the typical duration is between 10 and 25 sessions.
Other forms of psychotherapy exist, e.g., interpersonal therapy (IPT)  or cognitive-analytic therapy (CAT) . However, it is beyond the scope of this review to mention all new approaches to psychotherapy since the field is constantly expanding. Further, despite the existence of well-established manualized and evidence-based approaches to psychotherapy, a large proportion of practicing psychotherapists define themselves as eclective or integrative .
How the interventions might work
It is a common opinion among clinicians and researchers that patients suffering from complex psychiatric distress require longer and more intensive psychotherapy . Complex psychiatric distress can be defined as disorders, which by definition are enduring and inflexible , such as personality disorders or schizophrenia, chronic psychiatric disorders (defined as lasting at least a year), or multiple psychiatric disorders. A related assumption is that complex and severe problems typically take longer to improve than less complex or acute psychiatric distress [25, 64]. This is due to the inherent inflexibility of the psychopathology and the complexity of the required therapeutic techniques. Such potential therapeutic techniques could be provocation of affect or working with the therapeutic alliance . These are techniques that are potentially hard to carry out when faced with time constraints. However, it is often argued that such techniques are essential to effective psychotherapy .
In contrast, one could argue that long-term therapies can become counterproductive, given that the same therapeutic techniques will be repeated for a long period of time without continuous assessment of their effects. It is possible that given the limited therapeutic time, planned short-term psychotherapy forces both patients and therapists to establish and maintain a focus throughout the treatment process . Further, issues regarding termination of treatment are particularly important when conducting short-term psychotherapy, where concerns about termination are, almost by definition, always present [67, 68]. Thus, a possible advantage of short-term therapies is that both therapist and patient are forced to address difficult themes associated with separation and loss from the very beginning instead of postponing them for later.
Why is it important to do this review?
It is essential to investigate the optimal duration of psychotherapy for psychiatric disorders, because of the potential patient and health economic burden from long-term psychotherapy and because of the potential harmful effects of terminating treatment prematurely . If short-term psychotherapy is the optimal treatment approach, then this could result in a reduction of waitlists and thus a greater access to evidence-based care. On the contrary, if long-term psychotherapy is the most optimal treatment, then it becomes sensible for mental health systems to invest in these treatments, as they would translate into greater health and occupational benefits .
The relationship between the number of sessions (dose) and patient improvement (effect) in psychotherapy has previously been studied with mixed results [70, 71]. There are studies indicating that increased number of sessions is associated with diminishing results . There are also studies indicating that the speed of improvement is dependent on patients pretreatment functioning  and that some patients require different dosages to receive the same effect. However, most research on the association between dose and effect is based on uncontrolled studies [70,71,72, 74, 75] which can only show that patients improve during treatment. Whether this improvement can be attributed to the treatment, can only be established with randomized controlled trials, in which shorter and longer therapies are directly compared. A systematic review of such randomized clinical trials might allow us to assess the safety profile of the different treatment options directly. We are already aware of two randomized clinical trials comparing a short-term and a long-term version of the same psychotherapy type for one or more adult psychiatric disorders [76, 77]. We have performed a preliminary literature search in the Cochrane Database of Systematic Reviews (search terms, short-term or brief and long-term or standard psychotherapy) for previous systematic reviews comparing a short-term and a long-term version of the same psychotherapy type for one or more adult psychiatric disorders. We identified 1114 hits. From this preliminary literature search, we have only identified one empty systematic review .
The present systematic review aims at forming the basis for evidence-based guideline recommendations for the optimal duration of psychotherapy for adult psychiatric disorders taking bias risk (systematic errors), play of chance (random errors), and certainty of the findings into consideration. The objective of this review will be to assess the beneficial and harmful effects of short-term psychotherapy compared with long-term psychotherapy for adult psychiatric disorders.
The present protocol has been registered in the PROSPERO database (registration number, CRD42019128535) and is being reported in accordance with the reporting guidance provided in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) statement [79, 80] (see checklist in Additional file 1).
Criteria for considering studies for this review
Types of studies
We will include randomized clinical trials irrespective of trial design, setting, publication status, publication year, and language. We will not include quasi-randomized trials and observational studies.
Types of participants
Adults (as defined by trialists) with a primary diagnosis of any of the following psychiatric disorders: attention deficit hyperactivity disorder, psychotic disorders, depressive disorders, bipolar disorders, anxiety disorders, obsessive-compulsive disorder, trauma- and stressor-related disorders, eating disorders, and personality disorders, as defined by standardized diagnostic criteria from either ICD-10 , DSM-5 , or earlier versions (ICD-10 codes: F20–29, F30–39, F40–49, F50–59, F60–69, and F90–90.9). Participants will be included irrespective of sex and comorbidities.
Types of interventions
Experimental group: we will accept any type of short-term psychotherapy (or similar terms used by the trialists).
Control group: we will accept any type of long-term psychotherapy (or similar terms used by the trialists).
We will rely on the trialists defining their compared interventions as short-term and long-term (or similar terminology). We will include trials comparing a short-term and a long-term version of the same psychotherapy type (e.g., short-term psychodynamic therapy compared to long-term psychodynamic therapy). We will not include trials comparing short-term psychotherapy (e.g., short-term cognitive behavioral therapy) with a different type of psychotherapy (e.g., long-term psychodynamic therapy) delivered as long-term therapy. Further, we will include trials with the same dose (sessions) but with different frequencies, e.g., 12 sessions delivered over 6 weeks compared to 12 sessions delivered over 12 weeks.
Quality of life (continuous data)
Serious adverse events (dichotomous data). We will use the International Conference on Harmonisation of technical requirements for registration of pharmaceuticals for human use—Good Clinical Practice (ICH-GCP) definition of a serious adverse event, which is any untoward medical occurrence that resulted in death, was life-threatening, required hospitalization or prolonging of existing hospitalization and resulted in persistent or significant disability or jeopardized the patient . If the trialists do not use the ICH-GCP definition, we will include the data if the trialists use the term “serious adverse event.” If the trialists do not use the ICH-GCP definition nor use the term serious adverse event, then we will also include the data, if the event clearly fulfills the ICH-GCP definition for a serious adverse event.
Symptom severity assessed by any valid disease-specific symptom scale (continuous data). Symptoms will be analyzed separately for each disorder.
Suicide or suicide attempts as defined by trialists (dichotomous data)
Self-harm as defined by trialists (dichotomous data)
Level of functioning as defined by trialists (continuous data)
Assessment time points
The primary assessment time point will be the time point closest to the end of treatment in the trials’ long-term intervention group for all outcomes. For example, if a trial compares a 6-month and a 12-month version of the same psychotherapy type and outcomes are assessed every second month throughout the trial, we will select the assessment time point closest to the end of the 12-month intervention as the primary assessment time point for all outcomes. We will secondarily assess all outcomes at maximum follow-up if longer term follow-up is assessed.
Search methods for identification of studies
We will search Cochrane Central Register of Controlled Trials (CENTRAL), Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica database (EMBASE), Latin American and Caribbean Health Sciences Literature (LILACS), PsycINFO, Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Conference Proceedings Citation Index—Science (CPCI-S), and Conference Proceedings Citation Index—Social Science & Humanities (CPCI-SSH) to identify relevant trials. We will search all databases from their inception to the present. For a detailed search strategy for all electronic databases, see Additional file 2. The search strategy for PsycINFO will be given at the review stage.
Searching other resources
The reference lists of relevant publications will be checked for any unidentified randomized trials. We will contact the authors of included studies by email asking for unpublished randomized trials. Further, we will search for ongoing trials on the following:
Google Scholar (https://scholar.google.dk/)
The Turning Research into Practice (TRIP) Database (https://www.tripdatabase.com/)
European Medicines Agency (EMA) (http://www.ema.europa.eu/ema/)
US Food and Drug Administration (FDA) (www.fda.gov)
China Food and Drug Administration (CFDA) (http://eng.sfda.gov.cn/WS03/CL0755/)
Medicines and Healthcare Products Regulatory Agency (https://www.gov.uk/government/organisations/medicines-and-healthcare-products-regulatoryagency)
The World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) search portal (http://apps.who.int/trialsearch/)
Cochrane Database of Systematic Reviews
Additionally, we will hand search conference abstracts from psychiatry conferences for relevant trials. We will also consider relevant-for-the-review unpublished and gray literature trials if we identify these.
Data collection and analysis
We will perform the review following recommendations of the Cochrane Collaboration . The analyses will be performed using Trial Sequential Analysis  and Stata version 16 (StataCorp LLC, College Station, TX, USA) .
Selection of studies
Two authors (SJ and SS) will independently screen titles and abstracts. We will retrieve all relevant full-text study reports/publications, and two review authors (SJ and SS) will independently screen the full text and identify and record reasons for exclusion of the ineligible studies. We will resolve any disagreement through discussion, or if required, we will consult a third person (JCJ). Trial selection will be displayed in an adapted flow diagram as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement .
Data extraction and management
Two authors (SJ and SS) will independently extract data from included trials. Disagreements will be resolved by discussion with a third author (JCJ). We will assess duplicate publications and companion papers of a trial together to evaluate all available data simultaneously (maximize data extraction, correct bias assessment). We will contact the trial authors by email to specify any additional data, which may not have been reported sufficiently or at all in the publication.
We will extract the following data: bias risk components (as defined below), trial design (parallel, factorial, or crossover), number of intervention arms, length of follow-up, estimation of sample size, and inclusion and exclusion criteria.
Participant characteristics and diagnosis
We will extract the following data: number of randomized participants, number of analyzed participants, number of participants lost to follow-up/withdrawals/crossover, compliance with interventions, age range (mean or median), sex ratio, and type of psychiatric disorder.
We will additionally report the proportion of participants in the compared groups who receive psychotropic medication.
Short-term psychotherapy characteristics
We will extract the following data: short-term psychotherapy type, treatment duration, number of sessions (dose), session lengths (minutes), number of sessions per week, and treatment format.
Long-term psychotherapy characteristics
We will extract the following data: long-term psychotherapy type, treatment duration, number of sessions (dose), session lengths (minutes), number of sessions per week, and treatment format.
We will extract the following data: type of co-intervention, treatment duration of co-intervention, number of sessions (or dose), and treatment format.
All outcomes listed above will be extracted from each randomized clinical trial, and we will identify if outcomes are incomplete or selectively reported according to the criteria described later in “incomplete outcome data” bias domain and “selective outcome reporting” bias domain.
Funding of the trial and notable conflicts of interest of trial authors will be extracted, if available. We will note in the “Characteristics of included studies” table if outcome data were not reported in a usable way. Two review authors (SJ and SS) will independently transfer data into the Stata file . Disagreements will be resolved through discussion, or if required, we will consult with a third author (JCJ).
Assessment of risk of bias in included studies
We will use the instructions given in the Cochrane Handbook for Systematic Reviews of Interventions  in our evaluation of the methodology and hence the risk of bias of the included trials. We will evaluate the methodology in respect of the following:
Random sequence generation
Blinding of participants and treatment providers
Blinding of outcome assessment
Incomplete outcome data
Selective outcome reporting
Other risk of bias
Overall risk of bias
Random sequence generation
Low risk: If sequence generation was achieved using computer random number generator or a random number table. Drawing lots, tossing a coin, shuffling cards, and throwing dice will also be considered adequate if performed by an independent adjudicator.
Unclear risk: If the method of randomization was not specified, but the trial was still presented as being randomized
High risk: If the allocation sequence was not randomized or only quasi-randomized. These trials will be excluded.
Low risk: If the allocation of patients was performed by a central independent unit, on-site locked computer, identical-looking numbered sealed envelopes, or containers prepared by an independent investigator
Uncertain risk: If the trial was classified as randomized but the allocation concealment process was not described
High risk: If the allocation sequence was familiar to the investigators who assigned participants
Blinding of participants and treatment providers
Low risk: If the participants and the treatment providers were blinded to intervention allocation and this was described
Uncertain risk: If the procedure of blinding was insufficiently described
High risk: If blinding of participants and the treatment providers was not performed
Blinding of outcome assessment
Low risk of bias: If it was mentioned that outcome assessors were blinded and this was sufficiently described
Uncertain risk of bias: If it was not mentioned if the outcome assessors in the trial were blinded or the extent of blinding was insufficiently described
High risk of bias: If no blinding or incomplete blinding of outcome assessors was performed
Incomplete outcome data
Low risk of bias: If missing data were unlikely to make treatment effects depart from plausible values. This could be either (1) there were no drop-outs or withdrawals for all outcomes or (2) the numbers and reasons for the withdrawals and drop-outs for all outcomes were clearly stated and could be described as being similar to both groups. Generally, the trial is judged as at a low risk of bias due to incomplete outcome data if drop-outs are less than 5%. However, the 5% cut-off is not definitive.
Uncertain risk of bias: If there was insufficient information to assess whether missing data were likely to induce bias on the results.
High risk of bias: If the results were likely to be biased due to missing data either because the pattern of drop-outs could be described as being different in the two intervention groups or the trial used improper methods in dealing with the missing data (e.g., last observation carried forward).
Selective outcome reporting
Low risk of bias: If a protocol was published before or at the time the trial begun and the outcomes specified in the protocol were reported on
Uncertain risk of bias: If no protocol was published
High risk of bias: If the outcomes in the protocol were not reported on
Other risk of bias
Low risk of bias: If the trial appears to be free of other components (for example, academic bias or for-profit bias) that could put it at risk of bias
Unclear risk of bias: If the trial may or may not be free of other components that could put it at risk of bias
High risk of bias: If there are other factors in the trial that could put it at risk of bias (for example, authors conducted trials on the same topic, for-profit bias)
Overall risk of bias
Low risk of bias: The trial will be classified as overall “low risk of bias” only if all of the bias domains described in the above paragraphs are classified as low risk of bias.
High risk of bias: The trial will be classified as “high risk of bias” if any of the bias risk domains described above are classified as “unclear” or high risk of bias.
We will assess the domains “blinding of outcome assessment,” “incomplete outcome data,” and “selective outcome reporting” for each outcome result. Thus, we can assess the bias risk for each outcome assessed in addition to each trial. Our primary conclusions will be based on the results of our primary outcome results with overall low risk of bias. Both our primary and secondary conclusions will be presented in the summary of findings tables.
Differences between protocol and the review
We will conduct the review according to this published protocol and report any deviations from it in the “Differences between the protocol and the review” section of the systematic review.
Measures of treatment effect
We will calculate risk ratios (RRs) with 95% confidence interval (CI) for dichotomous outcomes, as well as the Trial Sequential Analysis-adjusted CIs (see below).
We will calculate the mean differences (MDs) and consider calculating the standardized mean difference (SMD) with 95% CI for continuous outcomes. We will also calculate trial sequential analysis-adjusted CIs (see below).
Dealing with missing data
We will, as the first option, contact all trial authors to obtain any relevant missing data (i.e., for data extraction and for assessment of risk of bias, as specified above).
We will not impute missing values for any outcomes in our primary analysis. In our sensitivity analyses (see paragraph below), we will impute data.
We will primarily analyze scores assessed at single time points. If only changes from baseline scores are reported, we will analyze the results together with follow-up scores . If standard deviations (SDs) are not reported, we will calculate the SDs using trial data, if possible. We will not use intention-to-treat data if the original report did not contain such data. We will not impute missing values for any outcomes in our primary analysis. In our sensitivity analysis (see paragraph below) for continuous outcomes, we will impute data.
Assessment of heterogeneity
We will primarily investigate forest plots to visually assess any sign of heterogeneity. We will secondly assess the presence of statistical heterogeneity by chi2 test (threshold P < 0.10) and measure the quantities of heterogeneity by the I2 statistic [87, 88]. We will investigate possible heterogeneity through subgroup analyses. We may ultimately decide that a meta-analysis should be avoided .
Assessment or reporting biases
We will use a funnel plot to assess reporting bias if ten or more trials are included. We will visually inspect funnel plots to assess the risk of bias. We are aware of the limitations of a funnel plot (i.e., a funnel plot assesses bias due to small sample size). From this information, we assess possible reporting bias. For dichotomous outcomes, we will test asymmetry with the Harbord test  if τ2 is less than 0.1 and with the Rücker test if τ2 is more than 0.1. For continuous outcomes, we will use the regression asymmetry test  and the adjusted rank correlation .
Unit of analysis issues
We will only include randomized clinical trials. For trials using crossover design, only data from the first period will be included [83, 92]. There will therefore not be any unit of analysis issues. We will not include cluster randomized trials.
We will undertake the meta-analysis according to the recommendations stated in the Cochrane Handbook for Systematic Reviews of Interventions , Keus et al. , and the eight-step assessment suggested by Jakobsen et al. . We will use the statistical software Stata version 16  to analyze data. We will assess our intervention effects with both random-effects meta-analyses  and fixed-effects meta-analyses . We will use the more conservative point estimate of the two . The more conservative point estimate is the estimate closest to zero effect. If the two estimates are similar, we will use the estimate with the widest CI. We assess a total of six primary and secondary outcomes, and we will therefore consider a P value of 0.014 or less as the threshold for statistical significance . We will investigate possible heterogeneity through subgroup analyses. Ultimately, we may decide that a meta-analysis should be avoided . We will use the eight-step procedure to assess if the thresholds for significance are crossed . Our primary conclusion will be based on results with low risk of bias . Where multiple trial arms are reported in a single trial, we will include only the relevant arms. If two comparisons are combined in the same meta-analysis, we will halve the control group to avoid double-counting . Trials with a factorial design will be included. In case of, e.g., a 2 × 2 factorial designed trial, the two groups receiving short-term interventions will be considered short-term control groups, while the two groups receiving long-term control interventions will be considered long-term control groups. If quantitative synthesis is not appropriate due to considerable heterogeneity or a small number of included trials, we will report the results in a narrative way.
Trial sequential analysis
Traditional meta-analysis runs the risk of random errors due to sparse data and repetitive testing of accumulating data when updating reviews. We wish to control the risks of type I errors and type II errors. We will therefore perform Trial Sequential Analysis on the outcomes, in order to calculate the required information size (that is, the number of participants needed in a meta-analysis to detect or reject a certain intervention effect) and the cumulative Z-curve’s breach of relevant trial sequential monitoring boundaries [84, 97,98,99,100,101,102,103,104]. A more detailed description of trial sequential analysis can be found in the trial sequential analysis manual  and at http://www.ctu.dk/tsa/.
For dichotomous outcomes, we will estimate the required information size based on the observed proportion of patients with an outcome in the control group (the cumulative proportion of patients with an event in the control groups relative to all patients in the control groups), a relative risk reduction of 20%, an alpha of 1.4% for all our outcomes, a beta of 20%, and the observed diversity as suggested by the trials in the meta-analysis. For continuous outcomes, we will in the trial sequential analysis use the observed SD, a mean difference of the observed SD/2, an alpha of 1.4% for all outcomes, a beta of 20%, and the observed diversity as suggested by the trials in the meta-analysis.
Subgroup analysis and integration of heterogeneity
We will perform the following subgroup analyses when analyzing the primary outcomes (quality of life, serious adverse events, and symptom severity).
High risk of bias trials compared to low risk of bias trials
Types of psychiatric disorders
Types of psychotherapy comparisons
Trials above and below the mean difference in intervention lengths
We will use the formal test for subgroup interactions in Stata .
To assess the potential impact of the missing data for dichotomous outcomes, we will perform the two following sensitivity analyses on both the primary and secondary outcomes.
“Best-worst-case” scenario: We will assume that all participants lost to follow-up in the short-term experimental group had no serious adverse event, had no suicides, had no suicide attempts, and had no self-harm and that all those participants lost to follow-up in the long-term control group did not survive, had a serious adverse event, had a suicide attempt, and had at least one episode of self-harm.
“Worst-best-case” scenario: We will assume that all participants lost to follow-up in the short-term control group did not survive, had serious adverse event, had a suicide attempt, and had at least one episode of self-harm and that all those participants lost to follow-up in the long-term control group have survived, had no serious adverse event, had no suicide attempts, and had no self-harm.
We will present results of both scenarios in our review. When analyzing quality of life, symptom severity, and level of functioning, a “beneficial outcome” will be the group mean plus two standard deviations (SDs) (we will secondly use one SD in another sensitivity analysis) of the group mean and a “harmful outcome” will be the group mean minus two SDs (we will secondly use one SD in another sensitivity analysis) of the group mean . To assess the potential impact of missing SDs for continuous outcomes, we will perform the following sensitivity analysis.
Where SDs are missing and it is not possible to calculate them, we will impute SDs from trials with similar populations and low risk of bias. If we find no such trials, we will impute SDs from trials with a similar population. As the final option, we will impute SDs from all trials.
We will present results of this scenario in our review. Other post hoc sensitivity analyses might be warranted if unexpected clinical or statistical heterogeneity is identified during the analysis of the review results .
“Summary of findings” table
We will create a summary of findings table using each of the prespecified outcomes (quality of life, serious adverse events, symptom severity, suicide and suicide attempts, self-harm, and level of functioning) We will use the five GRADE considerations (bias risk of the trials, consistency of effect, imprecision, indirectness, and publication bias) to assess the quality of a body of evidence as it relates to the studies which contribute data to the meta-analyses for the prespecified outcomes [94, 105,106,107]. We will assess imprecision using Trial Sequential Analysis. Otherwise, we will use methods and recommendations described in the Cochrane Handbook for Systematic Reviews of Interventions  using GRADEpro software. We will justify all decisions to downgrade the quality of studies using footnotes, and we will make comments to aid the reader’s understanding of the review where necessary. Firstly, we will present our results in the Summary of Findings table based on the results from the trials with low risk of bias, and secondly, we will present the results based on all trials.
This protocol aims at comparing the effects of short-term psychotherapy with the effects of long-term psychotherapy for common adult psychiatric disorders to determine the best length of treatment. The outcomes will be quality of life, serious adverse events, symptom severity, suicide or suicide attempts, self-harm, and level of functioning.
This protocol has a number of strengths. The predefined methodology is based on the Cochrane Handbook for Systematic Reviews of Interventions , the eight-step assessment suggested by Jakobsen et al. , Trial Sequential Analysis , and GRADE assessment [105,106,107]. Hence, this protocol considers both risks of random errors and risks of systematic errors. Another strength of this protocol is that we pragmatically compare two overall treatment strategies with each other, i.e., the results of this review will potentially reflect the effects of the two strategies in clinical everyday practice.
Our protocol also has some limitations. The primary limitation is the potential for large heterogeneity as a result of including all psychiatric disorders and all types of psychotherapy. Therefore, we may ultimately decide that a meta-analysis is not warranted. Further, psychotherapy always consists of multiple treatment elements and it is likely that different interventions have different effects. Hence, if we show a difference between the compared strategies, it will be difficult to conclude what exactly caused the difference in effect. To minimize this limitation, a number of subgroups are planned, but results of subgroup analyses should always be interpreted with great caution. Another limitation is the large number of comparisons which increase the risk of type 1 error. We have adjusted our thresholds for significance according to the number of primary outcomes, but as mentioned, we have also included multiple subgroup analyses. This large risk of type 1 error will be considered when interpreting the review results. Further, we expect that no trials will have blinded treatment providers and patients. Even though blinding of patients should be relatively easy, blinding of treatment providers is theoretically possible but much more difficult to carry out. Finally, we rely on the trialists defining their compared interventions as short-term and long-term (or similar terminology). Hence, we will not include trials comparing a short-term and a long-term version of the same psychotherapy type, if the trialists did not explicitly define their interventions with such terminology. Using trialists’ definitions of short-term and long-term psychotherapy potentially introduces problems with heterogeneity. However, we believe that our choice of methodology from a pragmatic point of view is the best solution there is. First, trialists often report poorly and often do not themselves use thresholds and important data might be excluded from our review if we demand exact definitions of lengths. Further, we do not expect to include many trials in this systematic review. Hence, relying on trialists definitions of short-term versus long-term psychotherapy may increase the number of trials being eligible for inclusion. Finally, we believe this pragmatic methodology will lead to the inclusion of the most relevant trials.
Availability of data and materials
Data sharing is not applicable to this protocol article. We will publish all data including code in the supplementary material of the systematic review.
Attachment-based family therapy
Acceptance and commitment therapy
Attention deficit hyperactivity disorder
Cognitive behavioral therapy
Cochrane Central Register of Controlled Trials
Conference Proceedings Citation Index—Science
Conference Proceedings Citation Index—Social Science & Humanities
Diagnostic and statistical manual of mental disorders, 5th edition
European Medicines Agency
Excerpta Medica database
The Grading of Recommendations Assessment, Development and Evaluation
International Classification of Diseases and Related Health Problems – 10th edition
International Conference on Harmonisation of technical requirements for registration of pharmaceuticals for human use – Good Clinical Practice
International Clinical Trials Registry Platform
Medical Literature Analysis and Retrieval System Online
Preferred reporting items for systematic review and meta-analysis
Preferred reporting items for systematic review and meta-analysis – protocols
International Prospective Register of Systematic Reviews
Post-traumatic stress disorder
Science index citation expanded
Standardized mean difference
Social Science Citation Index
Turning research into practice
World Health Organization
Wittchen HU, et al. The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol. 2011;21(9):655–79.
Barlow DH. The Oxford handbook of clinical psychology: updated edition. New York: Oxford University Press; 2014.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Arlington: American Psychiatric Pub; 2013.
Kessler RC, et al. The prevalence and correlates of adult ADHD in the United States: results from the National Comorbidity Survey Replication. Am J Psychiatry. 2006;163(4):716–23.
Fayyad J, et al. Cross-national prevalence and correlates of adult attention-deficit hyperactivity disorder. Br J Psychiatry. 2007;190:402–9.
Birnbaum HG, et al. Costs of attention deficit–hyperactivity disorder (ADHD) in the US: excess costs of persons with ADHD and their family members in 2000. Curr Med Res Opin. 2005;21(2):195–205.
Wittchen HU, Jacobi F. Size and burden of mental disorders in Europe-a critical review and appraisal of 27 studies. Eur Neuropsychopharmacol. 2005;15(4):357–76.
McGrath J, et al. Schizophrenia: a concise overview of incidence, prevalence, and mortality. Epidemiol Rev. 2008;30(1):67–76.
Chong HY, et al. Global economic burden of schizophrenia: a systematic review. Neuropsychiatr Dis Treat. 2016;12:357.
Merikangas KR, et al. Prevalence and correlates of bipolar spectrum disorder in the world mental health survey initiative. Arch Gen Psychiatry. 2011;68(3):241–51.
Dilsaver SC. An estimate of the minimum economic burden of bipolar I and II disorders in the United States: 2009. J Affect Disord. 2011;129(1–3):79–83.
Compton WM, et al. Changes in the prevalence of major depression and comorbid substance use disorders in the United States between 1991-1992 and 2001-2002. Am J Psychiatry. 2006;163(12):2141–7.
Greenberg PE, et al. The economic burden of adults with major depressive disorder in the United States (2005 and 2010). J Clin Psychiatry. 2015;76(2):155–62.
Kessler RC, et al. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):593–602.
DuPont RL, et al. Economic costs of anxiety disorders. Anxiety. 1996;2(4):167–72.
Kessler RC, et al. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):617–27.
Ruscio AM, et al. The epidemiology of obsessive-compulsive disorder in the National Comorbidity Survey Replication. Mol Psychiatry. 2010;15(1):53–63.
Olesen J, et al. The economic cost of brain disorders in Europe. Eur J Neurol. 2012;19(1):155–62.
Kessler RC, et al. Posttraumatic stress disorder in the National Comorbidity Survey. Arch Gen Psychiatry. 1995;52(12):1048–60.
Goldstein RB, et al. The epidemiology of DSM-5 posttraumatic stress disorder in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions-III. Soc Psychiatry Psychiatr Epidemiol. 2016;51(8):1137–48.
Koenen K, et al. Posttraumatic stress disorder in the world mental health surveys. Psychol Med. 2017;47(13):2260–74.
Hudson JI, et al. The prevalence and correlates of eating disorders in the National Comorbidity Survey Replication. Biol Psychiatry. 2007;61(3):348–58.
Stuhldreher N, et al. Cost-of-illness studies and cost-effectiveness analyses in eating disorders: a systematic review. Int J Eat Disord. 2012;45(4):476–91.
Soeteman DI, et al. The economic burden of personality disorders in mental health care. J Clin Psychiatry. 2008;69:259–65.
Crits-Christoph P, Barber JP. Long-term psychotherapy. In: Snyder CR, Ingram RE, editors. Handbook of psychological change: Psychotherapy processes & practices for the 21st century. New York: Wiley, Inc.; 2000. p. 455–73.
Smit Y, et al. The effectiveness of long-term psychoanalytic psychotherapy - a meta-analysis of randomized controlled trials. Clin Psychol Rev. 2012;32(2):81–92.
Leichsenring F, Rabung S. Effectiveness of long-term psychodynamic psychotherapy: a meta-analysis. JAMA. 2008;300(13):1551–65.
Kernberg OF, et al. Transference focused psychotherapy: overview and update. Int J Psychoanal. 2008;89(3):601–20.
Giesen-Bloo J, et al. Outpatient psychotherapy for borderline personality disorder: randomized trial of schema-focused therapy vs transference-focused psychotherapy. Arch Gen Psychiatry. 2006;63(6):649–58.
Bateman A, Fonagy P. Mentalization-based treatment for personality disorders: a practical guide. New York: Oxford University Press; 2016.
Abbass AA, et al. Short-term psychodynamic psychotherapies for common mental disorders. Cochrane Database Syst Rev. 2014;(7):CD004687.
Pavlov I. Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. New York: Oxford University Press; 1927.
Skinner BF. Science and human behavior. New York: Macmillan; 1953.
Beck AT. Cognitive therapy of depression. New York: Guilford press; 1979.
Beck AT. Depression: clinical, experimental, and theoretical aspects. Philadelphia: University of Pennsylvania Press; 1967.
Beck A, Rush A. A cognitive model of anxiety formation and anxiety resolution. Stress Anxiety. 1975;2:69–80.
Salkovskis PM. Obsessional-compulsive problems: a cognitive-behavioural analysis. Behav Res Ther. 1985;23(5):571–83.
Beck AT, Davis DD, Freeman A. Cognitive therapy of personality disorders. New York: Guilford Publications; 2015.
Fairburn CG. Cognitive behavior therapy and eating disorders. New York: Guilford Press; 2008.
Grant PM, et al. Randomized trial to evaluate the efficacy of cognitive therapy for low-functioning patients with schizophrenia. Arch Gen Psychiatry. 2012;69(2):121–7.
Sensky T, et al. A randomized controlled trial of cognitive-behavioral therapy for persistent symptoms in schizophrenia resistant to medication. Arch Gen Psychiatry. 2000;57(2):165–72.
Linehan MM, et al. Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder: a randomized clinical trial and component analysis. JAMA Psychiatry. 2015;72(5):475–82.
Farrell JM, Shaw IA, Webber MA. A schema-focused approach to group psychotherapy for outpatients with borderline personality disorder: a randomized controlled trial. J Behav Ther Exp Psychiatry. 2009;40(2):317–28.
Hayes SC. Acceptance and commitment therapy, relational frame theory, and the third wave of behavioral and cognitive therapies. Behav Ther. 2004;35(4):639–65.
Gilbert P. An introduction to compassion focused therapy in cognitive behavior therapy. Int J Cogn Ther. 2010;3(2):97–112.
Ruiz FJ. A review of acceptance and commitment therapy (ACT) empirical evidence: correlational, experimental psychopathology, component and outcome studies. Int J Psychol Psychol Ther. 2010;10(1):125.
Leaviss J, Uttley L. Psychotherapeutic benefits of compassion-focused therapy: an early systematic review. Psychol Med. 2015;45(5):927–45.
Rogers CR. A theory of therapy, personality and interpersonal relationships in the client-centred framework. In: Koch S, editor. Psychology: a study of science. Vol III, Formulations of the person in social context. New York: McGraw Hill; 1959. p. 184–256.
Perls F, Hefferline G, Goodman P. Gestalt therapy. Excitement and growth in the human personality. New York: Julian; 1951.
Yalom ID. Existential psychotherapy. Vol. 1. New York: Basic Books; 1980.
Greenberg LS, Van Balen R. The theory of experience-centered therapies. In: Greenberg LS, Watson JC, Lietaer G, editors. Handbook of experiential psychotherapy. New York: Guilford Press; 1998. p. 28–57.
Shinohara K, et al. Behavioural therapies versus other psychological therapies for depression. Cochrane Database Syst Rev. 2013;(10).
Paivio SC, Nieuwenhuis JA. Efficacy of emotion focused therapy for adult survivors of child abuse: a preliminary study. J Trauma Stress. 2001;14(1):115–33.
Lock J, le Grange D. Family-based treatment of eating disorders. Int J Eat Disord. 2005;37(S1):S64–7.
Diamond G, Siqueland L, Diamond GM. Attachment-based family therapy for depressed adolescents: programmatic treatment development. Clin Child Fam Psychol Rev. 2003;6(2):107–27.
Brestan EV, Eyberg SM. Effective psychosocial treatments of conduct-disordered children and adolescents: 29 years, 82 studies, and 5,272 kids. J Clin Child Psychol. 1998;27(2):180–9.
Lam DH. Psychosocial family intervention in schizophrenia: a review of empirical studies. Psychol Med. 1991;21(2):423–41.
Miklowitz DJ, et al. A randomized study of family-focused psychoeducation and pharmacotherapy in the outpatient management of bipolar disorder. Arch Gen Psychiatry. 2003;60(9):904–12.
O’Farrell TJ, Clements K. Review of outcome research on marital and family therapy in treatment for alcoholism. J Marital Fam Ther. 2012;38(1):122–44.
Stanton MD, Shadish WR. Outcome, attrition, and family–couples treatment for drug abuse: a meta-analysis and review of the controlled, comparative studies. Psychol Bull. 1997;122(2):170.
Cuijpers P, et al. Interpersonal psychotherapy for depression: a meta-analysis. Am J Psychiatr. 2011;168(6):581–92.
Ryle A, Poynton AM, Brockman BJ. Cognitive-analytic therapy: active participation in change: A new integration in brief psychotherapy. Oxford: Wiley; 1990.
Norcross JC. A primer on psychotherapy integration. In: Norcross JC, Goldfried MR, editors. Handbook of psychotherapy integration. New York: Basic Books; 2005. p. 3–23.
Critchfield KL, Benjamin LS. Principles for psychosocial treatment of personality disorder: summary of the APA Division 12 Task Force/NASPR review. J Clin Psychol. 2006;62(6):661–74.
Norcross J. Empirically supported therapy relationships. In: Norcross JC, editor. Psychotherapy relationships that work: Therapist contributions and responsiveness to patients. New York: Oxford University Press; 2002. p. 3–16.
Piper WE, et al. A comparative study of four forms of psychotherapy. J Consult Clin Psychol. 1984;52(2):268.
Hoyt MF. Therapist resistances to short-term dynamic psychotherapy. J Am Acad Psychoanal. 1985;13(1):93–112.
Bloom B. Planned short-term psychotherapies. In: Snyder CR, Ingram RE, editors. Handbook of psychological change: Psychotherapy processes & practices for the 21st century. New York: Wiley; 2000. p. 429–54.
Lilienfeld S. Psychological treatments that cause harm. Perspect Psychol Sci. 2007;2(1):53–70.
Cuijpers P, et al. How much psychotherapy is needed to treat depression? A metaregression analysis. J Affect Disord. 2013;149(1–3):1–13.
Castonguay LG, et al. Practice-oriented research: approaches and applications. Bergin Garfield’s Handb Psychother Behav Chang. 2013;6:85–133.
Howard KI, et al. The dose-effect relationship in psychotherapy. Am Psychol. 1986;41(2):159–64.
Lambert MJ, Hansen NB, Finch AE. Patient-focused research: using patient outcome data to enhance treatment effects. J Consult Clin Psychol. 2001;69(2):159.
Stulz N, et al. Dose–effect relationship in routine outpatient psychotherapy: does treatment duration matter? J Couns Psychol. 2013;60(4):593–600.
Kopta SM, et al. Patterns of symptomatic recovery in psychotherapy. J Consult Clin Psychol. 1994;62(5):1009–16.
Knekt P, et al. Randomized trial on the effectiveness of long-and short-term psychodynamic psychotherapy and solution-focused therapy on psychiatric symptoms during a 3-year follow-up. Psychol Med. 2008;38(5):689–703.
Lorentzen S, et al. Comparison of short- and long-term dynamic group psychotherapy: randomised clinical trial. Br J Psychiatry. 2013;203(3):280–7.
Naeem F, Farooq S, Kingdon D. Cognitive behavioural therapy (brief versus standard duration) for schizophrenia. Cochrane Database Syst Rev. 2015;(10).
Moher D, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1.
Shamseer L, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:7647.
WorldHealthOrganization, International classification of diseases, 10th revision (ICD-10). 2009.
International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, ICH harmonised guideline: integrated addemdum to ICH E6(R1): guideline for good clinical practice (ICH-GCP). 2015.
Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions 5.1. 0: The Cochrane Collaboration; 2011.
Copenhagen Trial Unit. TSA - trial sequential analysis. http://www.ctu.dk/tsa/. [Web page]. Accessed 10 July 2019.
StataCorp. Stata statistical software: release 16. College Station, TX: StataCorp LLC; 2019. http://www.stata.com
Moher D, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.
Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.
Higgins JP, et al. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557.
Harbord RM, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med. 2006;25(20):3443–57.
Egger M, et al. Bias in meta-analysis detected by a simple, graphical test. Bmj. 1997;315(7109):629–34.
Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50:1088–101.
Elbourne DR, et al. Meta-analyses involving cross-over trials: methodological issues. Int J Epidemiol. 2002;31(1):140–9.
Keus F, et al. Evidence at a glance: error matrix approach for overviewing available evidence. BMC Med Res Methodol. 2010;10(1):90.
Jakobsen JC, et al. Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods. BMC Med Res Methodol. 2014;14(1):120.
DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–88.
DeMets DL. Methods for combining randomized clinical trials: strengths and limitations. Stat Med. 1987;6(3):341–8.
Wetterslev J, et al. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. J Clin Epidemiol. 2008;61(1):64–75.
Brok J, et al. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses. J Clin Epidemiol. 2008;61(8):763–9.
Brok J, et al. Apparently conclusive meta-analyses may be inconclusive—trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses. Int J Epidemiol. 2008;38(1):287–98.
Thorlund K, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses? Int J Epidemiol. 2008;38(1):276–86.
Wetterslev J, et al. Estimating required information size by quantifying diversity in random-effects model meta-analyses. BMC Med Res Methodol. 2009;9(1):86.
Thorlund K, Anema A, Mills E. Interpreting meta-analysis according to the adequacy of sample size. An example using isoniazid chemoprophylaxis for tuberculosis in purified protein derivative negative HIV-infected individuals. Clin Epidemiol. 2010;2:57.
Thorlund K, et al., User manual for trial sequential analysis (TSA). http://www.ctu.dk/tsa/files/tsa_manual.pdf, 2011.
Imberger G, et al. False-positive findings in Cochrane meta-analyses with and without application of trial sequential analysis: an empirical review. BMJ Open. 2016;6(8):e011890.
Guyatt GH, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical research ed). 2008;336(7650):924–6.
Schünemann HJ, et al. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. Can Med Assoc J. 2003;169(7):677–80.
Guyatt GH, et al. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. 2011;64(4):380–2.
The expert help from Sarah Louise Klingenberg (Information Specialist, The Cochrane Hepato-Biliary Group, Copenhagen Trial Unit, Copenhagen, Denmark) in making the search strategy is hugely appreciated.
This study is funded by the Mental Health Services, Capital Region of Denmark Research Fund and TrygFoundation, grant no. 123488. The funding bodies will not be involved in design, collection, analysis, interpretation of data, and in writing up the manuscripts.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.