Research objectives
The primary objective of this systematic review is to evaluate the accuracy of ancillary tests for the neurological determination of death compared to the three accepted reference standards: (1) clinical diagnosis, (2) four-vessel angiography and (3) radionuclide imaging. This objective will be investigated using two different populations with different baseline risks of brain death: comatose patients and patients with NDD. Our secondary objective is to determine the prevalence of false negatives (patients not declared dead by ancillary testing when they are in fact dead) and false positives (patients declared dead by ancillary testing when they are in fact not dead). Finally, our tertiary objective is to determine the safety of the different ancillary tests used for NDD.
Research approach
The methods used for this systematic review will follow strict methodological standards based on the Cochrane Collaboration Diagnostic Accuracy Working Group’s [14] recommendations. Our approach will include, in addition to our research goals and objectives, a thorough process for study identification and selection and a descriptive quality assessment of included studies. The data analysis, presentation and interpretation will be adapted for this systematic review of diagnostic test accuracy [15].
Target condition and study participants
Primary studies of ancillary tests for NDD have been conducted in two different populations. In systematic reviews of diagnostic tests, it is important to consider the different pre-test probabilities since they can influence the estimation of the sensitivity and specificity of the studied test [16]. For NDD, since the specificity of a test is the most important parameter to estimate (that is, we do not want to declare death for someone who is not dead), it is very important to know how the test performs for a population of patients (that is, comatose patients) with characteristics similar to our target population (brain-dead patients). For this review, we will consider two groups of patients from: (1) studies of patients with a clinical diagnosis of brain death and (2) studies of comatose patients with a brain injury who underwent ancillary testing to diagnose brain death. Some studies will have included patients with various coma criteria with no NDD but for whom ancillary testing was performed for NDD. Therefore, we will include two types of study for this population type: studies where the same set of criteria for inclusion was used for the whole cohort (cohort accuracy studies) and studies where there was a different set of inclusion criteria for NDD cases compared to comatose controls (case–control accuracy studies) [16].
Reference standards
From our initial literature search, and based on current clinical practice and guidelines, we will consider studies that used one of the three reference gold standards for NDD [7]:
-
Clinical diagnosis: An established cause of brain injury, absence of confounding (as stated by the authors), coma, absence of brainstem reflexes, apnea. We are aware that clinical diagnosis criteria have changed over time and vary depending on the country where the study was conducted. If the clinical diagnosis is incomplete according to the aforementioned criteria, but the authors cite and explain their clinical diagnosis criteria, the study will be included in the review.
-
Four-vessel angiography: No intracranial blood flow (beyond carotid bifurcation or circle of Willis).
-
Radionuclide imaging: Hollow skull phenomenon, absence of intracranial uptake or equivalent.
In some jurisdictions, an ancillary test, such as an electroencephalogram or transcranial Doppler test, is mandatory as part of the clinical diagnosis [17]. In studies where an ancillary test was used as part of the clinical diagnosis, we will consider the combination of the clinical evaluation and the ancillary test as the clinical diagnosis. We will document the clinical diagnosis plus the ancillary testing as a reference and perform subgroup analyses at a later stage (by comparing the diagnosis to a clinical diagnosis that does not include an ancillary test).
Studied tests
We will investigate the following ancillary tests that have been proposed or are currently in use for NDD: four-vessel angiography, radionuclide imaging using any tracer, transcranial Doppler testing, electroencephalogram, evoked potentials, CT angiography, CT perfusion scan, magnetic resonance angiography, magnetic resonance perfusion, PET scan and xenon CT. Four-vessel angiography and radionuclide imaging will be included in the analysis as either reference standards or studied tests depending on the definition of the clinical diagnosis. All ancillary tests will be compared to one of our three reference standards (clinical diagnosis, four-vessel angiography and radionuclide imaging).
Type of studies
We will include all retrospective and prospective diagnostic test studies (including cross-sectional and cohort studies) and interventional studies. Our aim in synthesizing all available evidence is to provide not only estimates of diagnostic accuracy but also to highlight the clinical and methodological quality of the evidence as well as important gaps in the evidence.
Search strategy
We will develop a comprehensive search strategy with an information specialist trained in the conduct of systematic reviews. We will use MESH (or the EMTREE equivalent) terms and free-text terms for the included population, studied tests and reference tests that are as sensitive and inclusive as possible (Appendix 1). We will search the MEDLINE, EMBASE and Cochrane Central databases (including the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, the Cochrane Methodology Register, the Database of Abstracts of Reviews of Effects, the Health Technology Assessment Database and the NHS Economic Evaluation Database) from their inception to the date when the authors are ready for the final abstraction of the selected references. The reference lists of all published narrative reviews, systematic reviews and eligible studies will be searched for additional references.
Study screening and inclusion
We will use the title and the abstract to select the studies for inclusion after the comprehensive search has been performed. If no abstract is available, the full text will be obtained unless the title shows the study is clearly irrelevant. The same inclusion and exclusion criteria will be used at each screening step. For the first step, from the abstracts and titles, a single author will exclude studies clearly irrelevant to brain death or ancillary testing. A 10% random sample of all potential references will be validated by a second author for agreement. A second screening will then be performed by at least two independent reviewers for each study that was not excluded at the first step. Full text copies of relevant reports will be obtained for independent analysis by two reviewers, who will make the final inclusion decision. Disagreements will be solved by consensus and consultation with a third author.
Inclusion criteria
Population and target condition
Studied tests and reference standard
-
For studies where patients are selected based on clinical NDD, only one ancillary test will be compared to the clinical diagnosis as the reference standard.
-
For studies where NDD was not clinically diagnosed before conducting an ancillary test, the reference test for the studied ancillary test will have to be either radionuclide imaging or four-vessel angiography. If these criteria are not met (for example, if a study performs only an ancillary test that diagnoses brain death without comparing the diagnosis to an accepted reference test), the study will not be included.
-
For studies where patients are included based on their comatose state, the studied ancillary test has to be compared to at least one of the reference standards: clinical diagnosis, four-vessel angiography or radionuclide imaging.
Exclusion criteria
-
Studies from which we cannot obtain or calculate the true and false positive and negative rates from the text, appendices or after contacting the main authors.
-
Studies for which the objective was to determine diagnostic criteria of a specific ancillary test with no a priori definition of the diagnostic criteria to be used to define brain death.
-
Studies where only a single ancillary test was performed without any reference test.
-
Case reports (≤2 cases).
-
Duplicates or sub-cohorts of already published cohorts.
Analysis plan
For our primary and secondary objectives, a descriptive analysis of all included studies will be listed in tables and as text. The clinical, demographic and methodological quality characteristics of the reference and ancillary tests and the results will be reported and discussed. An in-depth discussion of the variability between studies will be provided where applicable. We will perform meta-analyses of suitable studies and discuss the results.
Study characteristics
Each study will be categorized as observational or interventional. Interventional studies will be categorized as randomized or non-randomized, and observational studies will be further categorized as prospective (cohort) or retrospective (cohort or case–control). Other characteristics of each study design will be reported. We will describe the population studied including the total number of patients, clinical characteristics, presence of confounding factors, the number of NDD patients, the number of patients in control groups where applicable and the characteristics of the patients (age, gender and reasons for brain death) and we will also list the inclusion and exclusion criteria and the criteria used for the reference and ancillary tests.
Risk of bias
We expect that most of the included studies will be observational in nature. Diagnostic meta-analyses of observational studies may be more prone to bias [15]. For the current systematic review, particular care will be taken to investigate the methodological and clinical risk of bias adequately before proceeding with the meta-analysis [15].
Methodological quality assessment
The methodological quality of the studies included in this systematic review will be evaluated by two independent reviewers using the QUADAS-2 tool, which is used to assess the quality of diagnostic tests [18]. QUADAS-2 is an updated version of this evidence-based quality tool. It is used in systematic reviews of diagnostic accuracy studies and is recommended by the Cochrane Collaboration Diagnostic Accuracy Working Group [14]. This tool helps to evaluate the principal methodological risk of bias in systematic reviews of diagnostic test accuracy [15]. Specific coding instructions adapted for this review will be included for the reviewers. Overall and study-specific appraisals of methodological strengths and weaknesses will be reported.
Clinical risk of bias
Although most countries use similar criteria for NDD [5], we may uncover differences in diagnosis practices for the clinical reference standards. Moreover, although clinical guidelines are available in most countries for NDD, the way they are applied can vary [10]. It is thus possible that clinical diagnosis criteria will vary across studies. We will very carefully define the criteria used for the clinical diagnosis of brain death for each included trial and if possible contact the authors for further details. A sensitivity analysis will be performed to evaluate the impact of the differences.
Similar issues may be observed for the diagnostic criteria used for ancillary testing. The diagnostic criteria for NDD for the ancillary tests used in each study will be collected and analyzed to investigate the differences between them. Technology has significantly changed over the years. For example, in radionuclide imaging, the tracers infused into patients for brain uptake have changed. In cases where the exact diagnostic criteria used for NDD in an ancillary test are not be available, we will attempt to contact the authors for details.
Primary analysis
For each study we will collect the rates for true positives, false positives, true negatives and false negatives and compute from the crude data the sensitivity and specificity, positive likelihood ratio and negative likelihood ratio. If possible and appropriate, parameter estimates will be computed using a bivariate model to produce summary receiver operating curves, summary operating points (pooled sensitivity and specificity) and 95% confidence regions around the summary operating point [19, 20]. These will be used to compare the accuracy of ancillary tests. These analyses will be performed for the following combinations:
-
1.
Clinical NDD patients
-
a.
a. Each ancillary test will be compared to the clinical diagnosis.
-
b.
Each ancillary test will be compared to four-vessel angiography or radionuclide imaging.
-
2.
Comatose patients
-
a.
a. Each ancillary test will be compared to the clinical diagnosis.
-
b.
Each ancillary test will be compared to four-vessel angiography or radionuclide imaging.
-
3.
For case–control diagnostic test studies, we will evaluate the possibility of pooling NDD groups and comatose groups to their respective population. Results before and after the inclusion of these patients will be reported.
Secondary and tertiary analyses
For the secondary analysis, we will report isolated brainstem death prevalence, false positive and false negative rates and a description of the cases. We will also report what the authors considered as confounding factors in their inclusion criteria for NDD. These data will be obtained from both prospective and retrospective studies. For our tertiary analysis, we will report the safety of ancillary tests. We will collect all safety-related events reported by the authors such as any delay in organ retrieval due to the use of an ancillary test, adverse events for the organ donor related to the use of contrast agents (such as renal failure and allergic reactions) and adverse events related to the transfer of unstable patients from an intensive care unit.
Planned subgroup analyses to explain heterogeneity
In meta-analysis of diagnostic test studies, heterogeneity is expected and univariate tests for heterogeneity are usually not performed [20]. To explore clinical and statistical heterogeneity, we will fit separate receiver operating curves for these planned subgroups of patients: (1) adults ≥18 years vs. mixed children/adults, only children vs. only adults and we will consider subgroups for the children (<1 month and neonates vs. children <18 years); (2) use of an ancillary test combined with clinical diagnosis as the reference standard, vs. clinical NDD alone, for a given ancillary test; (3) time after clinical diagnosis of brain death < or >24 hours; (4) studies that included patients with confounders (craniectomy, anoxic brain injury, hypothermia, barbiturates or other confounding factor as stated by the authors) vs. not included and (5) significant changes in ancillary test technology or diagnostic criteria.
Planned sensitivity analyses to study impact of study design on ancillary test accuracy
Based on guidance provided by the Cochrane Collaboration, we will use subgroups to explore and explain any heterogeneity in test accuracy and we will perform sensitivity analysis to investigate robustness [20]. It is impossible to identify all potential sensitivity analyses, but based on QUADAS-2 we can pre-specify important sensitivity analyses that will have to be performed: risk of selection bias, risk of interpretation bias for the studied ancillary test and risk of bias introduced by the interpretation of the reference test [18].