Limitations of evidence

Brain blood flow imaging, such as four-vessel angiography, and functional tests, such as radionuclide imaging, have traditionally been used as the gold standard ancillary tests for NDD [4]. Recently, several additional ancillary tests, such as computed tomography (CT) angiography, CT perfusion, magnetic resonance angiography and xenon CT, have been proposed as replacements for these traditional tests to confirm NDD [4] and their clinical use despite the absence of proper validation is growing [7, 9]. From a recent American survey, physicians used several different ancillary tests for the same patient and often used tests that were either not recommended or not validated [9]. Significant institutional and clinical practice variability thus remains in the use of ancillary tests, their indication and their diagnostic criteria [4, 9, 10]. Furthermore, when ancillary tests were applied on clinically brain-dead patients, a small proportion of the patients presented detectable brain perfusion, suggesting at best inaccuracy in the diagnostic criteria used for these ancillary tests and at worst a population that would have received a diagnosis of brain death based on current clinical criteria but in whom there is residual blood flow [11, 12]. Recently, the traditionally accepted accuracy of four-vessel angiography for NDD has even been challenged [13]. Although a recent Canadian expert consensus reported that the validation of ‘current or evolving techniques of brain blood flow imaging should not evolve into practice in a manner similar to cerebral angiography and radionuclide scanning, based on clinical application’, it is acknowledged that currently used ancillary tests and even traditionally accepted gold standards for NDD have not been subjected to rigorous evaluation [7]. The same group issued recommendations for more research to confirm the validity and accuracy of current ancillary reference tests and to develop new techniques for NDD [7].

Research objectives

The primary objective of this systematic review is to evaluate the accuracy of ancillary tests for the neurological determination of death compared to the three accepted reference standards: (1) clinical diagnosis, (2) four-vessel angiography and (3) radionuclide imaging. This objective will be investigated using two different populations with different baseline risks of brain death: comatose patients and patients with NDD. Our secondary objective is to determine the prevalence of false negatives (patients not declared dead by ancillary testing when they are in fact dead) and false positives (patients declared dead by ancillary testing when they are in fact not dead). Finally, our tertiary objective is to determine the safety of the different ancillary tests used for NDD.

Research approach

The methods used for this systematic review will follow strict methodological standards based on the Cochrane Collaboration Diagnostic Accuracy Working Group’s [14] recommendations. Our approach will include, in addition to our research goals and objectives, a thorough process for study identification and selection and a descriptive quality assessment of included studies. The data analysis, presentation and interpretation will be adapted for this systematic review of diagnostic test accuracy [15].

Target condition and study participants

Primary studies of ancillary tests for NDD have been conducted in two different populations. In systematic reviews of diagnostic tests, it is important to consider the different pre-test probabilities since they can influence the estimation of the sensitivity and specificity of the studied test [16]. For NDD, since the specificity of a test is the most important parameter to estimate (that is, we do not want to declare death for someone who is not dead), it is very important to know how the test performs for a population of patients (that is, comatose patients) with characteristics similar to our target population (brain-dead patients). For this review, we will consider two groups of patients from: (1) studies of patients with a clinical diagnosis of brain death and (2) studies of comatose patients with a brain injury who underwent ancillary testing to diagnose brain death. Some studies will have included patients with various coma criteria with no NDD but for whom ancillary testing was performed for NDD. Therefore, we will include two types of study for this population type: studies where the same set of criteria for inclusion was used for the whole cohort (cohort accuracy studies) and studies where there was a different set of inclusion criteria for NDD cases compared to comatose controls (case–control accuracy studies) [16].

Reference standards

From our initial literature search, and based on current clinical practice and guidelines, we will consider studies that used one of the three reference gold standards for NDD [7]:

In some jurisdictions, an ancillary test, such as an electroencephalogram or transcranial Doppler test, is mandatory as part of the clinical diagnosis [17]. In studies where an ancillary test was used as part of the clinical diagnosis, we will consider the combination of the clinical evaluation and the ancillary test as the clinical diagnosis. We will document the clinical diagnosis plus the ancillary testing as a reference and perform subgroup analyses at a later stage (by comparing the diagnosis to a clinical diagnosis that does not include an ancillary test).

Studied tests

We will investigate the following ancillary tests that have been proposed or are currently in use for NDD: four-vessel angiography, radionuclide imaging using any tracer, transcranial Doppler testing, electroencephalogram, evoked potentials, CT angiography, CT perfusion scan, magnetic resonance angiography, magnetic resonance perfusion, PET scan and xenon CT. Four-vessel angiography and radionuclide imaging will be included in the analysis as either reference standards or studied tests depending on the definition of the clinical diagnosis. All ancillary tests will be compared to one of our three reference standards (clinical diagnosis, four-vessel angiography and radionuclide imaging).

Type of studies

We will include all retrospective and prospective diagnostic test studies (including cross-sectional and cohort studies) and interventional studies. Our aim in synthesizing all available evidence is to provide not only estimates of diagnostic accuracy but also to highlight the clinical and methodological quality of the evidence as well as important gaps in the evidence.

Search strategy

We will develop a comprehensive search strategy with an information specialist trained in the conduct of systematic reviews. We will use MESH (or the EMTREE equivalent) terms and free-text terms for the included population, studied tests and reference tests that are as sensitive and inclusive as possible (Appendix 1). We will search the MEDLINE, EMBASE and Cochrane Central databases (including the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, the Cochrane Methodology Register, the Database of Abstracts of Reviews of Effects, the Health Technology Assessment Database and the NHS Economic Evaluation Database) from their inception to the date when the authors are ready for the final abstraction of the selected references. The reference lists of all published narrative reviews, systematic reviews and eligible studies will be searched for additional references.

Study screening and inclusion

We will use the title and the abstract to select the studies for inclusion after the comprehensive search has been performed. If no abstract is available, the full text will be obtained unless the title shows the study is clearly irrelevant. The same inclusion and exclusion criteria will be used at each screening step. For the first step, from the abstracts and titles, a single author will exclude studies clearly irrelevant to brain death or ancillary testing. A 10% random sample of all potential references will be validated by a second author for agreement. A second screening will then be performed by at least two independent reviewers for each study that was not excluded at the first step. Full text copies of relevant reports will be obtained for independent analysis by two reviewers, who will make the final inclusion decision. Disagreements will be solved by consensus and consultation with a third author.

Inclusion criteria

Exclusion criteria

Analysis plan

For our primary and secondary objectives, a descriptive analysis of all included studies will be listed in tables and as text. The clinical, demographic and methodological quality characteristics of the reference and ancillary tests and the results will be reported and discussed. An in-depth discussion of the variability between studies will be provided where applicable. We will perform meta-analyses of suitable studies and discuss the results.

Study characteristics

Each study will be categorized as observational or interventional. Interventional studies will be categorized as randomized or non-randomized, and observational studies will be further categorized as prospective (cohort) or retrospective (cohort or case–control). Other characteristics of each study design will be reported. We will describe the population studied including the total number of patients, clinical characteristics, presence of confounding factors, the number of NDD patients, the number of patients in control groups where applicable and the characteristics of the patients (age, gender and reasons for brain death) and we will also list the inclusion and exclusion criteria and the criteria used for the reference and ancillary tests.

Risk of bias

We expect that most of the included studies will be observational in nature. Diagnostic meta-analyses of observational studies may be more prone to bias [15]. For the current systematic review, particular care will be taken to investigate the methodological and clinical risk of bias adequately before proceeding with the meta-analysis [15].

Primary analysis

Secondary and tertiary analyses

For the secondary analysis, we will report isolated brainstem death prevalence, false positive and false negative rates and a description of the cases. We will also report what the authors considered as confounding factors in their inclusion criteria for NDD. These data will be obtained from both prospective and retrospective studies. For our tertiary analysis, we will report the safety of ancillary tests. We will collect all safety-related events reported by the authors such as any delay in organ retrieval due to the use of an ancillary test, adverse events for the organ donor related to the use of contrast agents (such as renal failure and allergic reactions) and adverse events related to the transfer of unstable patients from an intensive care unit.

Planned subgroup analyses to explain heterogeneity

In meta-analysis of diagnostic test studies, heterogeneity is expected and univariate tests for heterogeneity are usually not performed [20]. To explore clinical and statistical heterogeneity, we will fit separate receiver operating curves for these planned subgroups of patients: (1) adults ≥18 years vs. mixed children/adults, only children vs. only adults and we will consider subgroups for the children (<1 month and neonates vs. children <18 years); (2) use of an ancillary test combined with clinical diagnosis as the reference standard, vs. clinical NDD alone, for a given ancillary test; (3) time after clinical diagnosis of brain death < or >24 hours; (4) studies that included patients with confounders (craniectomy, anoxic brain injury, hypothermia, barbiturates or other confounding factor as stated by the authors) vs. not included and (5) significant changes in ancillary test technology or diagnostic criteria.

Planned sensitivity analyses to study impact of study design on ancillary test accuracy

Based on guidance provided by the Cochrane Collaboration, we will use subgroups to explore and explain any heterogeneity in test accuracy and we will perform sensitivity analysis to investigate robustness [20]. It is impossible to identify all potential sensitivity analyses, but based on QUADAS-2 we can pre-specify important sensitivity analyses that will have to be performed: risk of selection bias, risk of interpretation bias for the studied ancillary test and risk of bias introduced by the interpretation of the reference test [18].