Skip to main content

Non-invasive brain stimulation for treatment-resistant schizophrenia: protocol of a systematic review and network meta-analysis

Abstract

Background

Non-invasive brain stimulation (NIBS) is a promising intervention for treatment-resistant schizophrenia. However, there are multiple available techniques and a comprehensive synthesis of evidence is lacking. Thus, we will conduct a systematic review and network meta-analysis to investigate the comparative efficacy and safety of NIBS techniques as an add-on to antipsychotics for treatment-resistant schizophrenia.

Methods

We will include single- and double-blind randomized-controlled trials (RCT) comparing any NIBS technique with each other or with a control intervention as an add-on to antipsychotics in adult patients with treatment-resistant schizophrenia. We will exclude studies focusing on predominant negative symptoms, maintenance treatment, and single sessions. The primary outcome will be a change in overall symptoms, and secondary outcomes will be a change in symptom domains, cognitive performance, quality of life, functioning, response, dropouts, and side effects. We will search for eligible studies in previous reviews, multiple electronic databases and clinical trial registries from inception onwards. At least two independent reviewers will perform the study selection, data extraction, and risk of bias assessment. We will measure the treatment differences using standardized mean difference (SMD) and odds ratio (OR) for continuous and dichotomous outcomes, respectively. We will conduct pairwise and network meta-analysis within a frequentist framework using a random-effects model, except for rare event outcomes where we will use a fixed-effects Mantel–Haenszel method. We will investigate potential sources of heterogeneity in subgroup analyses. Reporting bias will be assessed with funnel plots and the Risk of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN) tool. The certainty in the evidence will be evaluated using the Confidence in Network Meta-analysis (CINeMA) approach.

Discussion

Our network meta-analysis would provide an up-to-date synthesis of the evidence from all available RCTs on the comparative efficacy and safety of NIBS for treatment-resistant schizophrenia. This information could guide evidence-based clinical practice and improve the outcomes of patients.

Systematic review registration

PROSPERO-ID CRD42023410645.

Peer Review reports

Background

Antipsychotic drugs are the mainstay treatment of schizophrenia [1], but a substantial number of patients meet the criteria for treatment-resistant schizophrenia and do not sufficiently respond to antipsychotics [2, 3], including clozapine [4]. In such cases, non-invasive brain stimulation (NIBS) techniques, such as electroconvulsive therapy (ECT), transcranial magnetic stimulation (TMS), and transcranial electric stimulation (tES), can be promising treatment options [5]. However, the evidence regarding the efficacy and safety of NIBS for treatment-resistant schizophrenia is currently inconclusive. Despite this, guidelines recommend considering ECT and TMS due to the high non-response rates to antipsychotics and the lack of other interventions [1, 6].

Existing syntheses of the evidence rely on systematic reviews and meta-analyses with mixed results, that included a few small clinical trials, did not specifically focus on treatment resistance, and applied a pairwise meta-analytic approach (e.g., investigating one pairwise comparison between interventions at a time) [7,8,9,10,11,12,13,14,15,16]. Therefore, they provided limited information about the comparative efficacy and safety among the different NIBS techniques and their findings are often difficult to interpret. For example, a Cochrane review found that adjunctive ECT was superior to usual care but not to sham interventions [7].

To overcome these limitations and better inform evidence-based management, we will conduct a network meta-analysis (NMA) to jointly analyze all available clinical trials that investigated NIBS techniques for treatment-resistant schizophrenia.

Methods/design

The protocol was reported according to the PRISMA statement for protocols [17] (eAppendix 1) and registered on 03.04.2023 with PROSPERO (ID: CRD42023410645) (eAppendix 4). The reporting of the systematic review will follow the PRISMA statement for network meta-analysis [18]. The eAppendix 2 provides information regarding the current status of the review and any changes made from the initial version of the protocol. If further amendments to the protocol are necessary, we will update the PROSPERO registration and include a clear report of any deviations in the published manuscript.

Eligibility criteria

Study design

We will include randomized trials (RCTs) comparing any NIBS to each other or a control condition as an add-on to antipsychotics for treatment-resistant schizophrenia, in which outcome assessors were blinded to the treatment (at least single-blind) [19]. We will exclude maintenance studies, in which patients were stabilized with NIBS before randomization. We will also exclude studies with a high risk of bias in the randomization process [20] and investigate monotherapy treatment or a single session with NIBS. If a trial is described as double-blind, but randomization is not explicitly mentioned, we will assume that the trial was randomized. In case of crossover studies, data from the first crossover phase will be used to avoid carry-over effects [19, 21]. Cluster-randomized trials will be included, and appropriate corrections in the estimations of the relative treatment effects will be applied [22]. There will be no other restrictions in terms of sample size, follow-up time, and country of origin [22].

Participants

Adult participants with a treatment-resistant form of schizophrenia, schizoaffective, or schizophreniform disorder will be eligible.

We will accept any study definition of treatment resistance since previous definitions varied widely across trials investigating NIBS [23] and did not fully align with the criteria of the Treatment Response and Resistance in Psychosis (TRRIP) group [2]. The different levels of stringency for defining treatment resistance will be classified as low, intermediate, and high cutoffs, similar to previous reviews [24, 25], and will be examined in subgroup analyses. Accordingly, studies requiring all participants to have treatment-resistant positive symptom domains (e.g., auditory hallucinations) will be eligible, e.g., [26], since positive symptoms have a central role in treatment-resistant schizophrenia [27]. Nevertheless, we will exclude studies in other specific populations, e.g., requiring all participants to have predominant negative symptoms, cognitive impairment, or comorbidities such as depression or drug abuse. In addition, we will assume that patients in the trials received treatment with antipsychotics in cases where it was not explicitly mentioned unless there is explicit information indicating otherwise, as monotherapy treatment with NIBS will be excluded.

There will be no additional restrictions in terms of age (adults-study defined, no upper age limit), setting, gender, ethnicity, severity of illness, and means of diagnosis (operationalized criteria or not). Studies including participants with other mental health conditions would be eligible only if at least 80% of the participants have a diagnosis of schizophrenia, schizoaffective, or schizophreniform disorder.

Experimental interventions

Any non-invasive brain stimulation (NIBS) administered as an add-on to antipsychotics will be eligible. There are currently multiple NIBS techniques that could be classified into four general modalities sharing neurophysiological mechanisms:

  1. 1.

    Electroconvulsive therapy (ECT) involves the induction of a seizure by administering electrical stimulus with electrodes placed in the scalp, typically under general anesthesia (modified-ECT) [28]. There are different protocols based on the location of electrodes (e.g., bilateral or unilateral, frontal or temporal) and electrical dosage.

  2. 2.

    Magnetic seizure therapy (MST) utilizes a magnetic field to induce the seizure and is considered more focal with fewer adverse effects than ECT [29].

  3. 3.

    Transcranial magnetic stimulation (TMS) can target distinct brain regions by administering electromagnetic pulses via coils [12, 30]. There are different protocols based on the frequency and pattern of pulses (e.g., repetitive TMS [rTMS] of low or high frequency, priming TMS when the high frequency is followed by low, theta burst stimulation [TBS] when 50 Hz bursts are administered at theta frequency, alpha-synchronized rTMS when stimulation is synchronized to the alpha frequency), focality and depth of stimulation (e.g., deep TMS with H-coils), location of coils (e.g., bilateral or unilateral, and prefrontal, tempoparietal, or cerebellar), and density of sessions (e.g., accelerated when multiple sessions are administered daily in order to condense rTMS within a shorter period of time).

  4. 4.

    Transcranial electrical stimulation (tES) involves the administration of weak electrical currents, usually via a bipolar electrode in the scalp [15]. There are different protocols based on the pattern of electrical stimulation (e.g., transcranial direct current stimulation [tDCS], transcranial alternating current stimulation [tACS] with a fixed frequency or transcranial random noise stimulation [tRNS]), and the location of the electrodes (e.g., bilateral or unilateral).

We will exclude NIBS monotherapy and single sessions, as well as other interventions, such as invasive brain stimulation including vagus nerve stimulation and deep brain stimulation, traditional medicine (e.g., acupuncture), psychotherapy, cognitive remediation, and lifestyle interventions. We will also exclude combination treatments such as NIBS combined with psychosocial or pharmacological intervention, except for adjunctive NIBS to treatment as usual with antipsychotic medications (see “Control interventions” section).

Control interventions

Any NIBS technique will be compared to each other and to control conditions, which could be classified into three main categories:

  1. 1.

    Sham interventions are procedures that simulate the different NIBS techniques in order to facilitate blinding and control for placebo effects [19]. They should be administered as adjunctive to treatment as usual with antipsychotic medications.

  2. 2.

    Treatment with antipsychotics without sham interventions or NIBS will be considered as treatment as usual (TAU), irrespective of the duration, the number, dose, and type of antipsychotics. Nevertheless, the initiation of a new antipsychotic as an add-on treatment to treatment as usual will not be eligible.

  3. 3.

    Other control conditions (e.g., waiting list) will be eligible if identified during the screening process.

Outcomes

Primary outcome

The primary outcome will be a change in overall symptoms of schizophrenia as measured by the Positive and Negative Syndrome Scale (PANSS) [31], the Brief Psychiatric Rating Scale (BPRS) [32], or any other validated scale [33]. PANSS and BPRS have been used in almost all schizophrenia trials [34], yet some trials investigating NIBS focused on positive symptom domains and did not utilize a score for overall symptoms [11, 16]. Therefore, when scores of overall symptoms will not be available, scores of positive symptoms will be used instead. This decision will allow a comprehensive synthesis of evidence, and will be investigated in a sensitivity analysis (see below “Sensitivity analyses” section).

Secondary outcomes

The secondary outcomes will be changes in quality of life, overall functioning, and symptom domains of schizophrenia as measured with validated scales, i.e., positive and negative symptoms, depressive symptoms, and cognitive performance. The cognitive performance will be investigated similarly to our previous analysis [35] and classified into global composite scores and scores for the seven domains of MATRICS [36], including attention/vigilance, speed of processing, working memory, visual learning, verbal learning, reasoning, and problem-solving as well as social cognition.

Moreover, we will examine the number of patients with a positive response to treatment (preferably defined as ≥ 20% reduction of PANSS or BPRS total scores [37], other cut-offs or study definitions will also be eligible), number of participants prematurely discontinued from the studies (i.e., dropouts due to any reason, inefficacy or adverse events), mortality due to any reason, the number of patients with serious adverse events [38], and the number of patients with specific side-effects such as neurological, cognitive, cardiovascular and musculoskeletal [12, 15, 28, 30].

Timing of outcome assessment

All outcomes will be assessed at the primary treatment endpoint of each study, which could range from a few days to weeks. There is evidence from antipsychotic trials in schizophrenia, that statistically significant separation between treatment and placebo response could be detected already in the first week, yet clear separation can be observed after at least three weeks [39]. However, the time course of treatment response with NIBS is unclear and can also vary across modalities as some studies demonstrated the efficacy of tES and TMS after five days of treatment [26, 40]. Therefore, we will accept any treatment endpoint (except after a single session) and we will extract data at all available time points for the primary outcome and at the treatment endpoint for the secondary outcomes. Given the potential relationship between treatment duration and the number of sessions, as well as their variation across different modalities and potential influence from other characteristics, we will attempt to a posteriori group the treatment duration and number of sessions of the treatments into meaningful categories to examine their impact on the primary outcome (see “Subgroup analyses” section).

Further, we will evaluate the primary outcome at several follow-up timepoints after the end of treatment (1, 3, 6, and 12 months) as secondary outcomes, since there is mixed evidence on the maintenance of any improvement after the end of treatment, e.g., [26, 41].

Information sources and search strategy

We will search multiple electronic databases without restrictions in terms of document type, publication status, publication period or language [22], i.e., EMBASE, PubMed, MEDLINE, PsycINFO, the clinical trials registers of the Cochrane Central Register of Controlled Trials (CENTRAL), ClinicalTrials.gov and WHO International Clinical Trials Registry Platform, and the three main Chinese databases of Wanfang Database, China National Knowledge Infrastructure (CNKI), and China Biology Medicine disc. The search terms will be developed together with experienced information specialists and methodologists (FS, JX). The search strategies on PubMed and CNKI are presented in eAppendix 3. We will also inspect reference lists of all included studies and previous reviews investigating NIBS for schizophrenia, e.g., [7,8,9,10,11,12, 14,15,16, 42]. In case of missing information, we will contact the first and/or corresponding author of included studies published in the last 30 years and companies manufacturing NIBS devices.

Study selection and data collection

Study selection

Two independent reviewers will screen identified titles/abstracts for inclusion, and disagreements will be resolved by discussion or by acquiring full articles for further inspection. Full texts of relevant titles/abstracts will be obtained, and in a second step, two independent reviewers will evaluate them against the eligibility criteria. Study selection will be reported in flow diagrams according to the PRISMA statement [18, 43] and we will provide a table listing the studies excluded at the full-text level, along with the respective reasons for exclusion. Disagreements will be resolved by discussion with a third senior reviewer, or contacting study authors. Records will be managed using Citavi [44].

Data extraction

Two independent reviewers will extract data on specifically developed forms in a Microsoft Access database that is tailor-made by our group for schizophrenia trials. Discrepancies in double data extraction will be identified by an algorithm, and doubts will be resolved by discussion with a third senior reviewer or by contacting study authors. We will extract information about study design and methodology, participant and intervention characteristics, and outcome measures. We will provide a table of study characteristics for the included studies.

For continuous outcomes, we will prefer change over endpoint scores, and methods accounting for missing outcome data (e.g., mixed-models of repeated measurement (MMRM), multiple imputations) over last-observation carried forward (LOCF) and over observed cases. Missing standard deviations (SD) will be derived from test statistics [22], by contacting study authors, or from SDs of other included studies using a validated imputation method [45].

For dichotomous outcomes, in case studies present only observed cases, we will assume that participants lost to follow-up had not responded to treatment or had not developed side effects. We judge this as appropriate since otherwise many rare side-effects would be overestimated. It is also a conservative approach regarding efficacy outcomes.

Risk of bias assessment

Two independent reviewers will evaluate the risk of bias of the primary outcome and dropouts due to any reason using the risk of bias tool RoB-2 [20], which considers the domains of the randomization process, deviations of indented interventions, missing outcome data, measurement of the outcome and selection of the reported result. Within-study reporting bias will additionally be evaluated with the Risk Of Bias due to Missing Evidence in Network meta-analysis (RoB-MEN) tool [46] (see below “Small-study effects and reporting bias” section). Discrepancies will be resolved by discussion with a third senior reviewer or by contacting study authors.

Data synthesis

Two-step procedure

We plan to conduct a network meta-analysis on the comparative efficacy and safety of NIBS. Network meta-analysis can combine direct and indirect evidence, simultaneously analyze all available clinical trials with increased precision and power, compare all interventions to each other, and provide treatment rankings [22].

We will follow a two-step procedure. First, we will perform a series of pairwise meta-analyses by investigating RCTs that compared directly two interventions. Second, if the requirements of NMA are met, we will conduct NMA in a frequentist framework [47]. We will use a random-effects model since heterogeneity is likely [48], and a fixed-effects Mantel–Haenszel method in case of rare dichotomous outcomes [49], such as mortality [50].

Effect-sizes

The effect size for continuous outcomes will be a standardized mean difference (SMD) since different rating scales are expected, and for dichotomous outcomes will be odds ratio (OR) because of their preferred mathematical properties [51, 52]. Effect sizes will be presented with their 95% confidence intervals. If heterogeneity is not large and treatment effects are estimated with comparable uncertainty, treatments will be ranked in the network meta-analysis using P-scores, the frequentist analog of the surface under the cumulative ranking curve (SUCRA) [53].

Network geometry

The network geometry will be presented with a network plot, in which nodes will represent different interventions and edges between nodes will represent the available trials that investigated a direct comparison between interventions. In the primary analysis, we will define nodes according to general modalities of NIBS, i.e., ECT, MST, TMS, and tES, and control conditions, i.e., sham interventions (reference comparator) and treatment as usual (Fig. 1). Thus, different NIBS protocols of the same modality and sham interventions will be merged. This decision is of clinical relevance due to the weak conceptual foundation of the majority of the protocols, and statistical power will be increased. Nevertheless, we will conduct sensitivity analyses by applying different definitions of nodes for NIBS at the level of a specific protocol, by defining distinct nodes for the sham interventions of the different modalities (e.g., sham-ECT, sham-TMS), and by classifying sham interventions into active (such as weak stimulation via coils applied with an angle of 45° or 90°) and inactive (such as inactive coils not producing magnetic fields) [54].

Fig. 1
figure 1

Theoretical network diagram for the primary analysis. The nodes represent the different interventions, and the edges connecting the different nodes represent the number of studies that compared directly two interventions. Their size will be proportionate to the number of participants and number of studies available, respectively

Transitivity assumption

We will include only trials in patients with treatment-resistant schizophrenia and exclude those in specific populations, and therefore, we will assume that patients in eligible trials are equally likely to be randomized to any of the interventions (i.e., transitivity assumption). The transitivity assumption is required for valid indirect comparisons and will be further explored by examining the distribution of potential effect modifiers across treatment comparisons [22], such as age, sex, baseline severity of symptoms, duration of illness, definition of treatment-resistance [24, 25], diagnosis (schizophrenia or schizoaffective disorder), rating scale, dose and type of antipsychotics, treatment duration and frequency of NIBS, blinding status, publication year, sample size, and sponsorship.

Assessment of heterogeneity

A common between-study variance (τ2) will be assumed across treatment comparisons within a network [22]. Heterogeneity will be quantified by comparing the τ2 with its empirical distributions [55, 56] and the magnitude will be classified into low, moderate, and high.

Assessment of incoherence

The statistical agreement between direct and indirect evidence will be evaluated within closed loops with the separating indirect from direct evidence approach (SIDE) and in the entire network with a design-by-treatment interaction test [22]. Tests of incoherence have low statistical power, and thus, sources of incoherence will be explored even in the absence of a statistically significant test result. In the presence of incoherence, we will explore analytical strategies, such as splitting the network into subgroups or using Bayesian network meta-regression to explore sources of incoherence and heterogeneity [22] (see below “Subgroup analyses” section).

Subgroup analyses

We will investigate potential sources of heterogeneity and/or incoherence in the primary outcome with subgroup analyses on (a) baseline severity of overall symptoms, (b) definition of treatment-resistance [24, 25], (c) duration of illness, (d) publication year, (e) sample size, (f) treatment duration, and (g) number of sessions. If network meta-regression is deemed appropriate and feasible for examining these potential effect modifiers, we will fit regressions in a Bayesian setting and assess the influence of modifiers by examining the credible intervals of the regression coefficients and evaluating changes in both heterogeneity and inconsistency between the unadjusted and adjusted models.

Sensitivity analyses

The robustness of the results for the primary outcome will be investigated with sensitivity analysis by excluding studies (a) that were only single-blind, (b) with an overall high risk of bias, (c) with implied randomization, (d) that did not use operationalized diagnostic criteria, (e) in which patients were assumed to receive antipsychotics if it was not clearly written in the study, (f) that required all patients to have treatment-resistant positive symptom domains, (g) with rating scales of positive symptoms used for the primary outcome, (h) imputed values, and (i) from mainland China [57], as well as by defining (j) different nodes for active and inactive sham interventions, (k) distinct nodes for sham interventions of different modalities and (l) different nodes for specific NIBS protocols.

Small-study effects and reporting bias

We will aim to include both published and unpublished RCTs. Small-study effects and the possible publication bias will be examined for the primary outcome (i.e., overall symptoms) and dropouts due to any reason with contour-enhanced funnel plots for pairwise meta-analysis when more than 10 studies are available [22], and comparison-adjusted funnel plots assuming the direction of bias towards newer interventions [58]. We will further evaluate reporting bias for the entire networks using the RoB-MEN tool [46], which considers within- and across-study assessments of reporting bias (see “Risk of bias assessment” section).

Confidence in the evidence

The confidence in the evidence will be evaluated for the primary outcome (i.e., overall symptoms) and dropouts due to any reason using the Confidence in Network Meta-Analysis (CINeMA) approach [59], considering within-study bias, reporting bias, indirectness, impression, heterogeneity, and incoherence.

Data analysis will be conducted in R statistical software [60] using the packages meta [61] and netmeta [62]. Bayesian network meta-regressions will be conducted with self-programmed routines in JAGS [63].

Discussion

Non-response to antipsychotics and treatment-resistance is a frequent condition in patients with schizophrenia [2], and it is associated with a poorer prognosis [64], reduced quality of life [65], and higher costs [65]. Among the limited number of interventions, non-invasive brain stimulation (NIBS) namely ECT and TMS, are recommended by current guidelines [1, 6], although the evidence is yet inconclusive and based on scattered pairwise meta-analyses, e.g., [7,8,9,10,11,12,13,14,15,16]. Our planned network meta-analysis would jointly synthesize the evidence from all available RCTs on the comparative efficacy and safety of NIBS techniques as an add-on to antipsychotics for treatment-resistant schizophrenia. Thus, this information could inform treatment guidelines and clinical practice [66].

There are potential challenges and limitations.

  • First, our network meta-analysis focuses on the acute phase trials investigating NIBS for treatment-resistance schizophrenia, and other potentially important aspects will not be considered, such as trials investigating maintenance treatment [67], predominant negative symptoms [68], and cognitive impairments [69]. However, we will explore these factors as secondary outcomes, specifically examining the different symptom domains of schizophrenia and the maintenance of symptom improvement following the completion of the treatment. Moreover, other intervention modalities that can be useful for treatment-resistant schizophrenia will not be considered, e.g., augmentation with another medication or psychological treatments [70].

  • Second, we will search Chinese databases and include studies from mainland China. Chinese databases represent a unique database for this research question and should not be disregarded, given the large number of available trials, and that ECT is widely used for schizophrenia in contrast to other countries [71]. However, studies from mainland China are often excluded from systematic reviews due to inadequate reporting and inappropriate randomization [57, 72]. Therefore, they will be carefully evaluated by native Chinese researchers and excluded in a sensitivity analysis.

  • Third, we will aim to analyze the safety of NIBS by considering dropouts due to adverse events, serious adverse events, and specific side effects. However, NIBS techniques can have distinct side-effect profiles, and thus, a long list of diverse side-effects is expected. In addition to this, the inconsistent reporting and assessment of side effects in clinical trials [73] may not allow a network meta-analysis for some outcomes, and in that case, a pairwise meta-analysis will be presented.

  • Last, it can be expected according to previous systematic reviews [7,8,9,10,11,12,13,14,15,16, 68] that there will be a few studies for comparison and that networks will be mainly star-shaped. In that case, heterogeneity and incoherence could not be well evaluated, and thus, we will downgrade accordingly the evidence in CINeMA and carefully interpret the findings.

In conclusion, our network meta-analysis could provide up-to-date information about the comparative efficacy and safety of non-invasive brain stimulation for treatment-resistant schizophrenia, which could guide evidence-based clinical practice and improve the outcomes of patients.

Availability of data and materials

Not applicable.

Abbreviations

BPRS:

Brief Psychiatric Rating Scale

CENTRAL:

Cochrane Central Register of Controlled Trials

CINeMA:

Confidence in Network Meta-Analysis

CNKI:

China National Knowledge Infrastructure

CSG:

Cochrane Schizophreni Group

ECT:

Electroconvulsive therapy

LOCF:

Last-observation carried forward

MCCB:

MATRICS Consensus Cognitive Battery

MMRM:

Mixed-models of repeated measurement

MST:

Magnetic seizure therapy

NIBS:

Non-invasive brain stimulation

NMA:

Network meta-analysis

OR:

Odds ratio

PANSS:

Positive and Negative Syndrome Scale

RCT:

Randomized-controlled trials

ROB-MEN:

Risk Of Bias due to Missing Evidence in Network meta-analysis

rTMS:

Repetitive transcranial magnetic stimulation

SD:

Standard deviations

SIDE:

Separating indirect from direct evidence approach

SMD:

Standardized mean difference

SUCRA:

Surface under the cumulative ranking curve

tACS:

Transcranial alternating current stimulation

TAU:

Treatment as usual

TBS:

Theta burst stimulation

tDCS:

Transcranial direct current stimulation

tES:

Transcranial electric stimulation

TMS:

Transcranial magnetic stimulation

TRRIP:

Treatment Response and Resistance in Psychosis

References

  1. Hasan A, Falkai P, Wobrock T, Lieberman J, Glenthoj B, Gattaz WF, Thibaut F, Möller HJ. World Federation of Societies of Biological Psychiatry (WFSBP) Guidelines for Biological Treatment of Schizophrenia, part 1: update 2012 on the acute treatment of schizophrenia and the management of treatment resistance. World J Biol Psychiatry. 2012;13(5):318–78.

    Article  PubMed  Google Scholar 

  2. Howes OD, McCutcheon R, Agid O, de Bartolomeis A, van Beveren NJ, Birnbaum ML, Bloomfield MA, Bressan RA, Buchanan RW, Carpenter WT, et al. Treatment-Resistant Schizophrenia: Treatment Response and Resistance in Psychosis (TRRIP) working group consensus guidelines on diagnosis and terminology. Am J Psychiatry. 2017;174(3):216–29.

    Article  PubMed  Google Scholar 

  3. Samara MT, Nikolakopoulou A, Salanti G, Leucht S. How many patients with schizophrenia do not respond to antipsychotic drugs in the short term? An analysis based on individual patient data from randomized controlled trials. Schizophr Bull. 2019;45(3):639–46.

    Article  PubMed  Google Scholar 

  4. Siskind D, Siskind V, Kisely S. Clozapine response rates among people with treatment-resistant schizophrenia: data from a systematic review and meta-analysis. Can J Psychiatry. 2017;62(11):772–7.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Kane JM, Agid O, Baldwin ML, Howes O, Lindenmayer JP, Marder S, Olfson M, Potkin SG, Correll CU. Clinical Guidance on the Identification and Management of Treatment-Resistant Schizophrenia. J Clin Psychiatry. 2019;80(2):18com12123.

    Article  PubMed  Google Scholar 

  6. DGPPN e.V. S3-Leitlinie Schizophrenie. Langfassung (Version 1.0) [S3 Guideline for Schizophrenia. Long version (Version 1.0)]. 2019. https://register.awmf.org/de/leitlinien/detail/038-009. Accessed 21 June 2024.

  7. Sinclair DJ, Zhao S, Qi F, Nyakyoma K, Kwong JS, Adams CE. Electroconvulsive therapy for treatment-resistant schizophrenia. Cochrane Database Syst Rev. 2019;3(3):Cd011847.

    PubMed  Google Scholar 

  8. Tharyan P, Adams CE. Electroconvulsive therapy for schizophrenia. Cochrane Database Syst Rev. 2005;2:Cd000076.

    Google Scholar 

  9. Wang G, Zheng W, Li X-B, Wang S-B, Cai D-B, Yang X-H, Ungvari GS, Xiang Y-T, Correll CU. ECT augmentation of clozapine for clozapine-resistant schizophrenia: a meta-analysis of randomized controlled trials. J Psychiatr Res. 2018;105:23–32.

    Article  PubMed  Google Scholar 

  10. Zheng W, Cao X-L, Ungvari GS, Xiang Y-Q, Guo T, Liu Z-R, Wang Y-Y, Forester BP, Seiner SJ, Xiang Y-T. Electroconvulsive therapy added to non-clozapine antipsychotic medication for treatment resistant schizophrenia: meta-analysis of randomized controlled trials. PLoS ONE. 2016;11(6):e0156510.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Dougall N, Maayan N, Soares-Weiser K, McDermott LM, McIntosh A. Transcranial magnetic stimulation (TMS) for schizophrenia. Cochrane Database of Systematic Reviews. 2015;2015(8):CD006081.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Lefaucheur JP, Aleman A, Baeken C, Benninger DH, Brunelin J, Di Lazzaro V, Filipović SR, Grefkes C, Hasan A, Hummel FC, et al. Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rTMS): An update (2014–2018). Clin Neurophysiol. 2020;131(2):474–528.

    Article  PubMed  Google Scholar 

  13. He H, Lu J, Yang L, Zheng J, Gao F, Zhai Y, Feng J, Fan Y, Ma X. Repetitive transcranial magnetic stimulation for treating the symptoms of schizophrenia: a PRISMA compliant meta-analysis. Clin Neurophysiol. 2017;128(5):716–24.

    Article  PubMed  Google Scholar 

  14. Kennedy NI, Lee WH, Frangou S. Efficacy of non-invasive brain stimulation on the symptom dimensions of schizophrenia: a meta-analysis of randomized controlled trials. Eur Psychiatry. 2018;49:69–77.

    Article  PubMed  Google Scholar 

  15. Fregni F, El-Hagrassy MM, Pacheco-Barrios K, Carvalho S, Leite J, Simis M, Brunelin J, Nakamura-Palacios EM, Marangolo P, Venkatasubramanian G. Evidence-Based Guidelines and Secondary Meta-Analysis for the Use of Transcranial Direct Current Stimulation in Neurological and Psychiatric Disorders. Int J Neuropsychopharmacol. 2021;24(4):256–313.

    Article  PubMed  Google Scholar 

  16. Cheng PWC, Louie LLC, Wong YL, Wong SMC, Leung WY, Nitsche MA, Chan WC. The effects of transcranial direct current stimulation (tDCS) on clinical symptoms in schizophrenia: A systematic review and meta-analysis. Asian J Psychiatr. 2020;53:102392.

    Article  PubMed  Google Scholar 

  17. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1–9.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, Ioannidis JP, Straus S, Thorlund K, Jansen JP, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162(11):777–84.

    Article  PubMed  Google Scholar 

  19. Brunoni AR, Fregni F. Clinical trial design in non-invasive brain stimulation psychiatric research. Int J Methods Psychiatr Res. 2011;20(2):e19-30.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng HY, Corbett MS, Eldridge SM, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366: l4898.

    Article  PubMed  Google Scholar 

  21. Elbourne DR, Altman DG, Higgins JP, Curtin F, Worthington HV, Vail A. Meta-analyses involving cross-over trials: methodological issues. Int J Epidemiol. 2002;31(1):140–9.

    Article  PubMed  Google Scholar 

  22. Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editor(s). Cochrane Handbook for Systematic Reviews of Interventions. 2nd ed. Chichester (UK): John Wiley & Sons; 2019.

  23. Kronick J, Sabesan P, Burhan AM, Palaniyappan L. Assessment of treatment resistance criteria in non-invasive brain stimulation studies of schizophrenia. Schizophr Res. 2022;243:349–60.

    Article  PubMed  Google Scholar 

  24. Samara MT, Dold M, Gianatsi M, Nikolakopoulou A, Helfer B, Salanti G, Leucht S. Efficacy, Acceptability, and Tolerability of Antipsychotics in Treatment-Resistant Schizophrenia: A Network Meta-analysis. JAMA Psychiat. 2016;73(3):199–210.

    Article  Google Scholar 

  25. Dong S, Schneider-Thoma J, Bighelli I, Siafis S, Wang D, Burschinski A, Schestag K, Samara M, Leucht S. A network meta-analysis of efficacy, acceptability, and tolerability of antipsychotics in treatment-resistant schizophrenia. Eur Arch Psychiatry Clin Neurosci. 2024;274(4):917–28.

    Article  PubMed  Google Scholar 

  26. Brunelin J, Mondino M, Gassab L, Haesebaert F, Gaha L, Suaud-Chagny MF, Saoud M, Mechri A, Poulet E. Examining transcranial direct-current stimulation (tDCS) as a treatment for hallucinations in schizophrenia. Am J Psychiatry. 2012;169(7):719–24.

    Article  PubMed  Google Scholar 

  27. Lee J, Takeuchi H, Fervaha G, Sin GL, Foussias G, Agid O, Farooq S, Remington G. Subtyping schizophrenia by treatment response: antipsychotic development and the central role of positive symptoms. Canadian journal of psychiatry Revue canadienne de psychiatrie. 2015;60(11):515–22.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Andrade C, Arumugham SS, Thirthalli J. Adverse effects of electroconvulsive therapy. Psychiatr Clin North Am. 2016;39(3):513–30.

    Article  PubMed  Google Scholar 

  29. Lisanby SH, Luber B, Schlaepfer TE, Sackeim HA. Safety and feasibility of magnetic seizure therapy (MST) in major depression: randomized within-subject comparison with electroconvulsive therapy. Neuropsychopharmacology. 2003;28(10):1852–65.

    Article  PubMed  Google Scholar 

  30. Rossi S, Antal A, Bestmann S, Bikson M, Brewer C, Brockmöller J, Carpenter LL, Cincotta M, Chen R, Daskalakis JD, et al. Safety and recommendations for TMS use in healthy subjects and patient populations, with updates on training, ethical and regulatory issues: Expert Guidelines. Clin Neurophysiol. 2021;132(1):269–306.

    Article  PubMed  Google Scholar 

  31. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13(2):261–76.

    Article  CAS  PubMed  Google Scholar 

  32. Overall JE, Gorham DR. The brief psychiatric rating scale. Psychol Rep. 1962;10(3):799–812.

    Article  Google Scholar 

  33. Marshall M, Lockwood A, Bradley C, Adams C, Joy C, Fenton M. Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia. Br J Psychiatry. 2000;176:249–52.

    Article  CAS  PubMed  Google Scholar 

  34. Huhn M, Nikolakopoulou A, Schneider-Thoma J, Krause M, Samara M, Peter N, Arndt T, Bäckers L, Rothe P, Cipriani A, et al. Comparative efficacy and tolerability of 32 oral antipsychotics for the acute treatment of adults with multi-episode schizophrenia: a systematic review and network meta-analysis. Focus (Am Psychiatr Publ). 2020;18(4):443–55.

    PubMed  Google Scholar 

  35. Feber L, Peter N, Schneider-Thoma J, Siafis S, Bighelli I, Hansen WP, Prates-Baldez D, Salanti G, Keefe RS, Engel RR, et al. Antipsychotic drugs and their effects on cognitive function: protocol for a systematic review, pairwise and network meta-analysis. Syst Rev. 2022;12(1):54.

    Article  Google Scholar 

  36. Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, Essock S, Fenton WS, Frese FJ 3rd, Gold JM, et al. The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. Am J Psychiatry. 2008;165(2):203–13.

    Article  PubMed  Google Scholar 

  37. Leucht S, Kane JM, Kissling W, Hamann J, Etschel E, Engel RR. What does the PANSS mean? Schizophr Res. 2005;79(2–3):231–8.

    Article  PubMed  Google Scholar 

  38. International Conference on Harmonisation Expert Working Group. ICH harmonised tripartite guideline. Clinical safety data management: definitions and standards for expedited reporting. IHC; 1994.

  39. Younis IR, Gopalakrishnan M, Mathis M, Mehta M, Uppoor R, Zhu H, Farchione T. Association of end point definition and randomized clinical trial duration in clinical trials of schizophrenia medications. JAMA Psychiat. 2020;77(10):1064–71.

    Article  Google Scholar 

  40. Poulet E, Brunelin J, Bediou B, Bation R, Forgeard L, Dalery J, d’Amato T, Saoud M. Slow transcranial magnetic stimulation can rapidly reduce resistant auditory hallucinations in schizophrenia. Biol Psychiatry. 2005;57(2):188–91.

    Article  PubMed  Google Scholar 

  41. Chang CC, Tzeng NS, Chao CY, Yeh CB, Chang HA. The effects of add-on fronto-temporal transcranial direct current stimulation (tDCS) on auditory verbal hallucinations, other psychopathological symptoms, and insight in schizophrenia: a randomized, double-blind, sham-controlled trial. Int J Neuropsychopharmacol. 2018;21(11):979–87.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Painuly N, Chakrabarti S. Combined use of electroconvulsive therapy and antipsychotics in schizophrenia: the Indian evidence. A review and a meta-analysis. J ect. 2006;22(1):59–66.

    Article  PubMed  Google Scholar 

  43. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906.

    Article  PubMed  Google Scholar 

  44. Swiss Academic Software. Citavi. https://www.citavi.com/en. Accessed 21 June 2024.

  45. Furukawa TA, Barbui C, Cipriani A, Brambilla P, Watanabe N. Imputing missing standard deviations in meta-analyses can provide accurate results. J Clin Epidemiol. 2006;59(1):7–10.

    Article  PubMed  Google Scholar 

  46. Chiocchia V, Nikolakopoulou A, Higgins JPT, Page MJ, Papakonstantinou T, Cipriani A, Furukawa TA, Siontis GCM, Egger M, Salanti G. ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis. BMC Med. 2021;19(1):304.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Rücker G. Network meta-analysis, electrical networks and graph theory. Res Synth Methods. 2012;3(4):312–24.

    Article  PubMed  Google Scholar 

  48. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–88.

    Article  CAS  PubMed  Google Scholar 

  49. Efthimiou O, Rücker G, Schwarzer G, Higgins JPT, Egger M, Salanti G. Network meta-analysis of rare events using the Mantel-Haenszel method. Stat Med. 2019;38(16):2992–3012.

    Article  PubMed  Google Scholar 

  50. Schneider-Thoma J, Efthimiou O, Huhn M, Krause M, Reichelt L, Röder H, Davis JM, Salanti G, Leucht S. Second-generation antipsychotic drugs and short-term mortality: a systematic review and meta-analysis of placebo-controlled randomised controlled trials. Lancet Psychiatry. 2018;5(8):653–63.

    Article  PubMed  Google Scholar 

  51. Doi SA, Furuya-Kanamori L, Xu C, Lin L, Chivese T, Thalib L. Controversy and Debate: Questionable utility of the relative risk in clinical research: Paper 1: A call for change to practice. J Clin Epidemiol. 2022;142:271–9. https://doi.org/10.1016/j.jclinepi.2020.08.019.

  52. Bakbergenuly I, Hoaglin DC, Kulinskaya E. Pitfalls of using the risk ratio in meta-analysis. Res Synth Methods. 2019;10(3):398–419.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Rücker G, Schwarzer G. Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Med Res Methodol. 2015;15:58.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Dollfus S, Lecardeur L, Morello R, Etard O. Placebo response in repetitive transcranial magnetic stimulation trials of treatment of auditory hallucinations in schizophrenia: a meta-analysis. Schizophr Bull. 2016;42(2):301–8.

    Article  PubMed  Google Scholar 

  55. Rhodes KM, Turner RM, Higgins JP. Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data. J Clin Epidemiol. 2015;68(1):52–60.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JPT. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. Int J Epidemiol. 2012;41(3):818–27.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Tong Z, Li F, Ogawa Y, Watanabe N, Furukawa TA. Quality of randomized controlled trials of new generation antidepressants and antipsychotics identified in the China National Knowledge Infrastructure (CNKI): a literature and telephone interview study. BMC Med Res Methodol. 2018;18(1):96.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Chaimani A, Higgins JPT, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta-analysis in STATA. PLoS ONE. 2013;8(10):e76654–e76654.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Nikolakopoulou A, Higgins JPT, Papakonstantinou T, Chaimani A, Del Giovane C, Egger M, Salanti G. CINeMA: an approach for assessing confidence in the results of a network meta-analysis. PLoS Med. 2020;17(4):e1003082.

    Article  PubMed  PubMed Central  Google Scholar 

  60. R Core Team . R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2021. https://www.R-project.org/. Accessed 21 June 2024.

  61. Balduzzi S, Rücker G, Schwarzer G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health. 2019;22(4):153–60.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Balduzzi S, Rücker G, Nikolakopoulou A, Papakonstantinou T, Salanti G, Efthimiou O, Schwarzer G. netmeta: An R Package for Network Meta-Analysis Using Frequentist Methods. J Stat Softw. 2023;106(2):1–40. https://doi.org/10.18637/jss.v106.i02.

  63. Plummer M. JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Proceedings of the 3rd international workshop on distributed statistical computing. 2003. (Vol. 124, No. 125.10, pp. 1–10).

  64. Andreasen NC, Arndt S, Alliger R, Miller D, Flaum M. Symptoms of schizophrenia. Methods, meanings, and mechanisms. Arch Gen Psychiatry. 1995;52(5):341–51.

    Article  CAS  PubMed  Google Scholar 

  65. Kennedy JL, Altar CA, Taylor DL, Degtiar I, Hornberger JC. The social and economic burden of treatment-resistant schizophrenia: a systematic literature review. Int Clin Psychopharmacol. 2014;29(2):63–76.

    Article  PubMed  Google Scholar 

  66. Leucht S, Chaimani A, Cipriani AS, Davis JM, Furukawa TA, Salanti G. Network meta-analyses should be the highest level of evidence in treatment guidelines. Eur Arch Psychiatry Clin Neurosci. 2016;266(6):477–80.

    Article  PubMed  Google Scholar 

  67. Chanpattana W. Maintenance ECT in schizophrenia: a pilot study. J Med Assoc Thai. 1998;81(1):17–24.

    CAS  PubMed  Google Scholar 

  68. Tseng PT, Zeng BS, Hung CM, Liang CS, Stubbs B, Carvalho AF, Brunoni AR, Su KP, Tu YK, Wu YC, et al. Assessment of noninvasive brain stimulation interventions for negative symptoms of schizophrenia: a systematic review and network meta-analysis. JAMA Psychiat. 2022;79(8):770–9.

    Article  Google Scholar 

  69. Smith RC, Md WL, Wang Y, Jiang J, Wang J, Szabo V, Faull R, Jin H, Davis JM, Li C. Effects of transcranial direct current stimulation on cognition and symptoms in Chinese patients with schizophrenia✰. Psychiatry Res. 2020;284:112617.

    Article  PubMed  Google Scholar 

  70. Wagner E, Kane JM, Correll CU, Howes O, Siskind D, Honer WG, Lee J, Falkai P, Schneider-Axmann T, Hasan A. Clozapine combination and augmentation strategies in patients with schizophrenia—recommendations from an international expert survey among the Treatment Response and Resistance in Psychosis (TRRIP) Working Group. Schizophr Bull. 2020;46(6):1459–70.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Tang YL, Jiang W, Ren YP, Ma X, Cotes RO, McDonald WM. Electroconvulsive therapy in China: clinical practice and research on efficacy. J ect. 2012;28(4):206–12.

    Article  PubMed  Google Scholar 

  72. Leucht S, Li C, Davis JM, Bighelli I, Zhu Y, Furukawa TA. About the issue of including or excluding studies from China in systematic reviews. Schizophr Res. 2022;240:162–3.

    Article  PubMed  Google Scholar 

  73. Pope A, Adams C, Paton C, Weaver T, Barnes TR. Assessment of adverse effects in clinical studies of antipsychotic medication: survey of methods used. Br J Psychiatry. 2010;197(1):67–72.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. Farhad Shokraneh and Jun Xia who developed the search strategies and conducted the first search in conventional and Chinese electronic databases, respectively.

Funding

Open Access funding enabled and organized by Projekt DEAL. This work is funded by the German Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung/BMBF) under the grand number 01KG2206. The funder was not involved in the design of the study, in the collection of data, in data analysis or interpretation, or in manuscript writing. The study was registered prospectively.

Author information

Authors and Affiliations

Authors

Contributions

SL supervises the study. SL and SS obtained the funding. SS, CL, FP, GS, and SL designed the systematic review and meta-analysis. W-PH provided the patients’ perspective. SS wrote the first draft of the manuscript. All authors critically reviewed the manuscript for important intellectual content and approved the final manuscript.

Corresponding author

Correspondence to Spyridon Siafis.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

FP is a member of the European Scientific Advisory Board of Brainsway Inc., Jerusalem, Israel, and the International Scientific Advisory Board of Sooma, Helsinki, Finland. He has received speaker’s honoraria from Mag&More GmbH and the neuroCare Group, Munich, Germany, as well as Brainsway Inc., Jerusalem, Israel. His lab has received support with equipment from the neuroCare Group, Mag&More GmbH and Brainsway Inc.

SL has received in the last 3 years honoraria as a consultant and/or advisor and/or for lectures and/or for educational material from Alkermes, Angelini, Eisai, Gedeon Richter, Janssen, Lundbeck, Medichem, Medscape, Merck Sharpp and Dome, Mitshubishi, Neurotorium, NovoNordisk, Otsuka, Recordati, Roche, Rovi, Sanofi Aventis, TEVA.

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

13643_2024_2585_MOESM1_ESM.pdf

Additional file 1: eAppendix 1. PRISMA-P. eAppendix 2. Current status and modifications from the first version of the PROSPERO registration. eAppendix 3. Search strategies. eAppendix 4. PROSPERO Registration CRD42023410645.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Siafis, S., Lorenz, C., Wu, H. et al. Non-invasive brain stimulation for treatment-resistant schizophrenia: protocol of a systematic review and network meta-analysis. Syst Rev 13, 165 (2024). https://doi.org/10.1186/s13643-024-02585-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-024-02585-2

Keywords