This systematic review protocol follows the Preferred Reporting Items for Systematic reviews and Meta-Analysis Protocol (PRIMSA-P) guideline (see Additional file 1) [21]. In addition, the protocol has been registered on the International Prospective Registry of Systematic Reviews PROSPERO (Registration number CRD42021205552).
Eligibility criteria
Population
We will include studies in patients with hematologic malignancies. Studies in patients with solid tumor malignancies will not be included. In the previous systematic review of CAR-T cell therapy by members of our group, there was no signal for efficacy in studies of solid tumor malignancies (complete response of 4.1 %, 95% CI, 1.6 to 10.6%) [12]; therefore, the decision was made to focus on studies of hematologic malignancy.
Intervention
Our intervention is chimeric antigen receptor T cell therapy. There will be no restrictions based on therapeutic dose, the number of therapeutic administrations, CAR-T target, CAR-T co-stimulatory domain, etc.
Comparator
Studies with or without comparators will be included.
Outcomes
The primary outcome of interest is complete response. Secondary outcomes include overall response, disease progression, relapse, overall survival, and adverse events (cytokine release syndrome, neurotoxicity, B cell aplasia, graft-versus-host disease). Tertiary outcomes of interest include patient experience, health-related quality of life, and health utility measures.
Study design
We will include all interventional studies. Observational studies (i.e., retrospective cohort studies, case report, case series, etc.) will be excluded.
Information sources
Searches will be conducted in the MEDLINE (OVID interface, including In-Process and Epub Ahead of Print), Embase (OVID interface), and the Cochrane Central Register of Controlled Trials (Wiley interface) databases from October 2017 to present. This represents the date at which the search from our previous review was last run [12]. We will update the search before publishing the full systematic review.
Search strategy
The literature search used in our previous review will be used as a guide [12]. The field of CAR-T cell therapy is rapidly evolving, and therefore new search terms will need to be developed and added to the search. This new search will be developed in conjunction with an information specialist and will be specific to hematologic malignancy patients. Prior to the execution of the full search strategy, a Peer Review of Electronic Search Strategies (PRESS) will be performed [22]. This represents an additional layer of methodological rigor, helping to ensure the accuracy of our search strategy. We will also search reference lists of included studies and relevant reviews identified in our search. The systematic review team will then be provided with a bibliography of included articles for feedback. A sample search strategy can be found in the online supplement (Additional file 2).
Study selection process
Results of the literature search will be uploaded to Distiller Systematic Review Software (DistillerSR®, Evidence Partners, Ottawa). The titles and abstracts will be independently reviewed by two reviewers using the predefined inclusion criteria as described above. To refine the screening question and to establish a high level of inter-rater correlation (kappa > 0.8), a calibration test involving ten studies will be performed before screening begins formally. The full text will be accessed for all titles that meet the inclusion criteria or in case of uncertainty. Two review authors will assess the full reports for eligibility. A third-party member will be consulted in order to resolve disagreements. The reasons for excluding studies will be recorded. For non-English full text articles, we will contract Ottawa Hospital faculty or employees with fluency in the language of the article to assist in determining article eligibility. The study selection process will be documented and reported using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram.
Data extraction process
Data extraction forms will be developed in DistillerSR® by the study team in order to capture all relevant information from each included study. Two reviewers will extract all relevant data from the included studies using this extraction form. Five studies will be used for calibration, and the review team will also be consulted for guidance if the data extraction form needs to be revised. Any disagreements will be reconciled through consensus and, if necessary, a third party will make a final decision. Where uncertainty is identified, or data of interest is not reported, study authors will be contacted for more information.
Data items and individual patient data extraction
We will collect basic study characteristics (i.e., first author, year of publication, country of trial, trial design, sample size), study-level patient data (sex, age, comorbidities, cancer type, previous lines of treatment, treatment dose, follow-up period, etc.), and intervention characteristics will be similar to our previous review [12] (manufacturing and cell product characteristics, including fresh or frozen, T cell origin, selection of T cell subsets, T cell expansion method including cell culture duration, CAR target antigen, CAR molecular structure [e.g., affinity domain, hinge domain, transmembrane domain, costimulatory domain(s), signaling domain], transfection/transduction method and the therapeutic regimen [CAR-T cell dose, frequency, duration, route of administration]). Outcome data and risk of bias information (both described below) will also be extracted. In addition, we will collect the same information listed above but at the individual patient level.
Dealing with obtaining IPD when it is not reported
The process of obtaining not-reported IPD can be lengthy and complex [23, 24], and failure to include IPD within an IPDMA can compromise the review’s objective, decrease statistical power, and introduce potential biases [25, 26]. Therefore, significant effort will be made to obtain not-reported IPD. Our review team will contact the corresponding author and/or study sponsor for each included clinical trial where IPD is not reported. We will follow recently published practical guidance on obtaining and managing individual patient data datasets [27]. In the case of non-response or refusal, data from these clinical trials will be included in the study-level analysis only. A complete list of all the data items listed above will be sought from each included clinical trial. Upon identification of not-reported IPD from each trial, it will be clearly outlined to corresponding authors what data we are asking for. If corresponding authors express interest in collaborating and providing data, they will be invited to read the protocol, and a virtual meeting will be scheduled to discuss details of sharing IPD. If IPD is provided, the corresponding author of the trial will be invited to be listed as a collaborator on the manuscript. Any data provided will be anonymized and stored on a secured server. Any remaining IPD that we fail to obtain will not be imputed and will be classified as missing. Also, if there is missing data for patients, we will classify them as missing and not impute for missing values.
Outcomes
Our primary outcome is complete response. Complete response definitions will vary, given our broad range of malignancies being included in the review. Complete response definitions will be collected. We will report the best overall response if a complete response is not possible. Complete response data will not include studies involving patients who were in complete remission at the time they started CAR-T cell therapy. The complete response data from the latest timepoint will be used. For example, we anticipate the identification of multiple publications from singular trials (i.e., longer-term follow-up). In this case, complete response rates at the longest follow-up provided will be extracted and analyzed. The duration of the complete response will be collected. Secondary outcomes include overall response, progression-free survival, relapse, overall survival, and adverse events (cytokine release syndrome, neurotoxicity, B cell aplasia, graft-versus-host disease). Overall response will be defined as the sum of partial and complete responses. Tertiary outcomes of interest include patient experience, health-related quality of life, and health utility measures.
Risk of bias assessment
We will use a modified version of the Institute of Health Economics tool to assess risk of bias [28, 29]. This modified tool was successfully implemented in previous reviews consisting of predominately single arm studies [12, 30]. As we have previously described, the tool evaluates the following elements: study objective, design, study population, intervention and cointerventions, outcome measures, statistical analysis, results, and conclusions, as well as conflicts of interest. The risk of bias for any controlled or comparative studies will be evaluated using the appropriate risk of bias assessment tool for randomized or non-randomized comparative trials recommended by Cochrane [31]. Risk of bias details will be extracted in duplicate by two independent reviewers, with any disagreements being resolved by a third party.
Data analysis
Study-level analysis
Our data analysis will consist of two approaches. First, we will perform the traditional, aggregate data approach. Briefly, dichotomous data (i.e., complete response) will be analyzed and presented in a proportion (truncated between 0 and 1) with 95% confidence intervals. Using Comprehensive Meta-Analysis (version 3; Biostat Inc., USA), we will use a random-effects model (DerSimonian and Laird) to pool outcomes. The Cochrane I2 statistic will be used to assess statistical heterogeneity. Alternative funnel plots (specific to single-arm studies) will be used to assess publication bias [32]. The quality of evidence will be assessed by GRADE (Grading of Recommendations, Assessment, Development and Evaluations) [33].
Individual patient data meta-analysis
Second, we will perform an IPDMA, which will be used to explore the effect of patient-level characteristics on our identified outcome measures. For the IPDMA, a one-stage approach will be used, which synthesizes the IPD from all included studies in a single step, while simultaneously accounting for the clustering of patients within studies [13]. Specifically, we will use a generalized linear mixed model with reduced random intercept and random slope which assumes the random effects to be independent. The one-stage approach was chosen, as it allows for more sophisticated modeling to explore our effect moderators of interest as opposed to pooling trial level estimates. By using this method, the exact binomial distribution is modeled, correlations within the study are taken into account which allows for greater flexibility in the model specification, and continuity corrections are not required [34]. A sensitivity analysis will be conducted using the two-stage approach, to assess the robustness of our analysis. The investigation of patient-level factors which may be acting as effect modifiers are described below in the subgroup analysis section.
Subgroup analyses
Next, we will perform subgroup analyses at the patient-level, as subgroup exploration at the patient-level is the primary purpose of our study. We will explore the effects of patient-level covariates on the efficacy (complete response) and safety (adverse events) of CAR-T cell therapy. Covariates of interest include sex, age, cancer type, cancer stage, previous number of treatments, use of lymphodepletion, use of bridging therapy, follow-up period, duration of response, CAR-T dose, number of CAR-T cell infusions, time between leukapheresis and infusion, CAR-T source (allogeneic vs autologous), CAR-T formulation, and co-stimulatory domain used. These covariates of interest have been chosen based on findings from our previous systematic review of CAR-T cell therapy, as well as discussions with clinical experts on the review team [12, 35]. Cancer type and CAR-T formulation are considered core covariates and will be included in all modeling. For assessing interactions, we will undertake a one-stage meta-analysis with a generalized linear model to estimate treatment-covariate interaction with both random treatment and interaction effects. The assumptions of such a model are that the true effect of the treatment varies between studies, the true effect of the interaction varies between studies, and that the random effects for both treatment and interaction are uncorrelated. Study-level subgroup analyses will be performed in the event that a significant portion of IPD is unable to be obtained. Study-level subgroup analyses include type of malignancy, age group (pediatric or adult), T cell origin (autologous or allogeneic), and CAR-T construct (e.g., CD19 or CD20, etc.).
Ethics and dissemination
Institutional ethics approval is not required for this study. Results from this review will be published in a peer-reviewed scientific journal and presented at appropriate scientific conferences. The final reporting of this manuscript will follow the guidance set forth by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses IPD statement [36]. Finally, our diverse team of collaborators and knowledge users will aide with the dissemination of study findings. In order to maximize impact of our study results, we have assembled a nationally representative team of knowledge-users. The Leukemia & Lymphoma Society of Canada (LLSC) has partnered with our team on prior CAR-T related research projects [37]. This partnership proved fruitful and they have once again agreed to act as a knowledge user for this proposed project. Cell Therapy Transplant Canada (CCTC) is a national organization within Canada, representing physicians, allied healthcare providers, and researchers working in the fields of hematopoietic cell transplantation and cellular therapy. They also have a diverse network of patients and patient’s families and are expertly positioned to act as knowledge users for this proposed research. Prior to study start-up, we will have a teleconference between team members and knowledge users to summarize our protocol and make any final adjustments based on reviewer comments. We will also identify any further stakeholders that may be interested. At the commencement of each new step in the project (i.e., screening, data extraction, etc.), we will have teleconference meetings to track progress. To meet the needs of our knowledge users, feedback will be continually sought in order to ensure that stakeholders are contributing in a meaningful manner with respect to the design and execution of our two major objectives including whether they feel their input is being heard and integrated.
Patient and public involvement
In developing this protocol, patients and members of the public were not involved. However, we have an ongoing relationship with two patient partners for CAR-T research, and we plan to involve our patient partners going forward in the project, specifically around the interpretation and disseminate of results. Our group has had a fruitful relationship with patient partners in the past for CAR-T-related research [37], involving them at many different stages of various projects.