Protocol for a systematic review of the diagnostic and prognostic utility of tests currently available for the detection of aspirin resistance in patients with established cardiovascular or cerebrovascular disease

Background The benefits of aspirin as an anti-platelet agent are well established; however, there has been much debate about the lack of uniformity in the efficacy of aspirin to inhibit platelet function. In some patients, aspirin fails to inhibit platelets even where compliance has been verified, a phenomenon which has been termed “aspirin resistance”. These patients may in turn be at a higher risk of future vascular events. The proportion of “resistant” patients identified depends on the type of platelet function test. Therefore, the aim of this systematic review is to determine which, if any, platelet function test has utility in terms of identifying patients with a high risk of vascular events. The review has been registered with PROSPERO (CRD42012002151). Methods Relevant studies will be sought from bibliographic databases. Trials registers will be searched for ongoing studies. Reference lists will be checked and subject experts contacted. There will be no date or language restrictions. Standard reviewing methodology to minimise bias will be employed. Any prospective studies in patients on aspirin therapy and assessing platelet function in relation to relevant clinical outcomes will be included, as will studies reporting prognostic models. Risk of bias assessment will be based on the Quality Assessment of Diagnostic Accuracy Studies guidelines, and suitable criteria for assessing quality of prognostic studies. Data on test accuracy measures, relative risks, odds or hazard ratios will be extracted and meta-analysed, where possible, using a random-effects model to account for between-study heterogeneity. Where appropriate, the causes of heterogeneity will be explored through meta-regression and sub-group or sensitivity analyses. If platelet function testing is demonstrated to have diagnostic/predictive utility in a specific population, the potential for a cost-effectiveness analysis will be considered and, if possible, an economic model constructed. This will be supported by a systematic review of existing economic evaluation studies. Discussion The results of the review could indicate if platelet function test(s) could lead to a reliable prediction of the risk of clinically important events in a defined population, and thus support investigations into adjustments to therapy in order to compensate for a predicted poor response to standard aspirin.


Background
The clinical efficacy of aspirin as an antiplatelet agent to prevent occlusive arterial events in patients with atherothrombotic disease is well established; daily administration of aspirin has been shown to reduce the risk of stroke, myocardial infarction (MI) and death by approximately 25% [1]. Despite these proven benefits, many patients continue to experience thrombotic events. Several factors may influence the response of platelets to antiplatelet therapy [2], including treatment adherence [3]. However, in some patients even in the context of verified compliance, aspirin fails to inhibit platelets as determined by ex vivo laboratory tests, a phenomenon termed "resistance" to antiplatelet therapy [4].
Many platelet function tests have been used to assess inhibition of platelet function induced by aspirin and their methodologies are diverse [5,6]. While some assays study global haemostasis, most platelet function assays target a specific phase of platelet function, from platelet adhesion to platelet activation, secretion and aggregation [7,8]. Methodological differences make it difficult to compare the test results. For example, some of these assays are carried out in whole blood (including whole blood aggregometry, platelet counting, PFA-100, VerifyNow, Impact-R and flow cytometry), while others require sample preparation (such as plasma or serum thromboxane B 2 measurement), and others can be performed on urine (levels of the thromboxane B 2 metabolite 11-dehydro-thromboxane B 2 ) [9]. Thresholds to determine whether platelet function is within a normal range may vary between studies of the same test depending on how the threshold was established (e.g., thresholds commonly reported in the literature, prescribed by the manufacturer or derived from comparisons with healthy control populations).
Most importantly, some assays are specific to the action of aspirin, by using an agonist that targets the platelet pathway inhibited by aspirin (the cyclooxygenase pathway) or by measuring metabolites of thromboxane A 2 (which is the end-product of the pathway blocked by aspirin), while other platelet function assays assess pathways not directly affected by aspirin but measure global platelet reactivity [5,10,11]. The effect of aspirin is, of course, more obvious in aspirin-specific assays, but the global platelet reactivity assays could be more clinically relevant as they capture a more global portrait of platelet function [12]. However, for such global assays to be able to determine resistance to aspirin therapy would require aspirin to be the only platelet function modifying agent used by the patient.
There is no official guideline recommending one assay above another, and platelet function testing is not recommended for routine clinical testing in patients requiring aspirin therapy [13]. As a result, many of the available platelet function assays have been used in a research capacity, and part of the uncertainty surrounding the definition and clinical relevance of aspirin resistance is due to the non-interchangeable nature of these assays [14]. As a consequence, there is a need to address basic questions on the prognostic and diagnostic utility and cost-effectiveness of platelet function testing in the context of aspirin therapy before testing can be recommended in clinical practice.

Existing systematic reviews
Three relevant recent systematic reviews were identified, all of which included a meta-analysis [15][16][17]. The review by Reny et al. [17] looked at the PFA-100 test only to predict cardiovascular events in patients with symptomatic atherosclerosis, and included 8 prospective studies. A statistically significant result was reported for an increased risk of ischemic events in patients defined as 'aspirin resistant' by the test. The other reviews included a range of platelet function tests and identified 17 studies [15] and 20 studies [16] respectively. Both reviews reported a statistically significant increased risk of cardiovascular events in patients with 'aspirin resistance' as defined by the laboratory test(s).
There were some methodological limitations associated with the latter two reviews, including a lack of detail on quality assessment of included studies, the thresholds (cut-off points) used to define 'aspirin resistance' for each test, and patient compliance. The diagnostic utility of the tests (e.g., in terms of sensitivity and specificity) was also not considered. Further, heterogeneity between included studies due to differences in platelet function assays, study follow-up periods and aspirin dose, needs to be accounted for and, where possible, investigated in more detail. Since the completion of these reviews, several further studies have been published, particularly due to the emergence of dual/triple antiplatelet therapy regimens (where aspirin is one agent) for treating some at-risk populations with cardiovascular disease, such as those with acute coronary syndrome or undergoing percutaneous coronary intervention.
The aim of the current project is to perform a new systematic review that, in comparison to existing reviews, additionally incorporates any missed studies and/or studies published more recently; implements a more robust and complete meta-analysis of the evidence, including an improved approach to quantifying and examining clinical and statistical heterogeneity; and summarises the evidence in relation to both prognostic association (e.g., odds ratios, hazard ratios) and diagnostic/predictive test accuracy (e.g., sensitivity, specificity) for individual patients.

Research aims
This systematic review will look at studies which relate platelet function testing to the risk of adverse clinical outcome(s) in patients on aspirin therapy with established cardiovascular or cerebrovascular disease, or diabetes. More specifically, the review will aim to determine the diagnostic/predictive utility and the prognostic utility of different platelet function tests: i) Diagnostic/predictive utility: to establish whether any of the available platelet function tests to determine "aspirin resistance" has sufficiently high diagnostic/predictive utility (e.g., sensitivity, specificity and positive and negative predictive values close to 1) in order to determine, for individual patients, if treatment modification should be considered based on the test result; ii) Prognostic utility: to establish whether any of the available platelet function tests has prognostic ability, i.e., whether it is able to distinguish between groups of patients with different average outcome risks even if it does not accurately predict individual outcome risk.
Further, the potential for a cost-effectiveness analysis will be considered. An economic model will be constructed and the potential for populating the model with data based on the results of the systematic review explored.

Types of study
Any prospective primary studies, or systematic reviews of such studies, assessing platelet function test(s) in relation to clinical outcomes.

Types of participants
Patients aged ≥18 years on aspirin (as monotherapy or in combination with other antiplatelet agents), with established cardiovascular or cerebrovascular disease, or diabetes. Studies with mixed populations will be included as long as data for relevant patients is extractable. Studies with patients on aspirin for peripheral vascular disease will be noted.

Setting
Studies in any setting will be included.

Technology
Either aspirin specific platelet function test or global platelet function test where patients are receiving aspirin as the only antiplatelet therapy; or aspirin specific platelet function test where patients are on dual/ triple antiplatelet therapy (with aspirin as one of the agents).

Outcomes
Clinical outcomes, such as: vascular events (non-fatal and fatal ischemic stroke, transient ischemic attack, systemic embolism (pulmonary embolism, peripheral arterial embolism), MI, stent thrombosis); mortality due to vascular events. Other outcomes will include: All-cause mortality; bleeding (categorised according to the International Society of Haemostasis and Thrombosis as major bleeding events, clinically relevant non-major bleeding events or minor bleeding events); major adverse cardiovascular events (composite of allcause mortality, non-fatal MI and stroke); revascularisation procedures (percutaneous coronary intervention, coronary arterial bypass graft, embolectomy).
ii) Prognostic model studies There may be studies that report prognostic models, in which a platelet function test is one of multiple prognostic factors predicting clinical outcomes in a population of interest. In order to examine the contribution of the test to the overall performance of the prognostic model, and to establish whether predictive accuracy of clinical outcomes is improved by combining test results with other prognostic factors, the studies will need to meet the following inclusion criteria: Reference lists of all included papers will be checked and subject experts will be contacted. No restrictions on publication language and date of publication will be applied.
In addition, abstracts from the following national and international proceedings will be consulted from 2009 onwards in order to capture studies that are not yet fully published:

Study selection
This will be a two-step process. Titles (and abstracts where available) will initially be screened by two reviewers, using pre-specified screening criteria. These are broadly based on whether the studies 1) are about platelet function testing, 2) report clinical outcomes, and 3) are in patients on aspirin therapy. Full texts of any potentially relevant articles will then be obtained and two reviewers will independently apply the full inclusion criteria. Any discrepancies between reviewers will be resolved by discussion or by referral to a third reviewer. Appropriate portions of non-English language articles will be translated where necessary. The study selection process will be illustrated using a PRISMA flow diagram [18]. Reference management software (Reference Manager v11 Thomson ResearchSoft) will be used to record reviewer decisions, including reasons for exclusion.

Data extraction
Data extraction will be conducted independently by two reviewers using a standardised, piloted data extraction form. Disagreements will be resolved through discussion or referral to a third reviewer.
Data extraction will include the following variables:  (see Table 1 below), otherwise relative risks, odds ratios or hazard ratios (with estimates of uncertainty [19]); where sufficient statistical information cannot be extracted, study authors will be asked to supply necessary information; Statistical methods employed and appropriateness of these.
Data will also be extracted on any prognostic models, including results of models reporting prognostic risk with and without test results as a parameter.

Assessment of risk of bias
Risk of bias will be assessed independently by two reviewers in accordance with Quality Assessment of Diagnostic Accuracy Studies (revised tool) (QUADAS-2) guidelines for diagnostic test studies [20]. As there is also a prognosis element, these guidelines will be supplemented with suitable criteria for checking the quality of prognostic studies as suggested by Hayden et al. [19]. Specific elements to be considered include: Study population (adequately described, details on sampling/recruitment); Study attrition (length of follow-up time, amount and timing of censoring, reasons for loss to follow-up); Test measurement (adequately described, timing of tests, missing tests results); Outcome measurement (adequately and unambiguously described, measured blind to the test result); Statistical analysis (whether continuous tests results were analysed on their continuous scale and, if not, whether a non-data-dependent explanation was given for the choice of thresholds to categorise the tests).
If any prognostic models are identified, the quality criteria described by Altman et al. [21] will be used in addition to those of Hayden et al. [19]. Specific elements to be considered include: Methods of model development (selection of candidate risk variables, relative weighting, handling of continuous variables); Internal and external model validations; Study design (prospective/retrospective); Sample size (considered a priori); Missing data (quantity, and how missing data was handled in the statistical analysis); Criteria for inclusion of prognostic factors into the model (adequately described, and whether wellknown prognostic factors were included regardless of significance).
Any prognostic models identified will be summarised qualitatively (such as included variables; calculation of risk score; predictive accuracy; whether the model was validated internally and externally) and quantitatively by extracting performance statistics for calibration (such as Observed/Expected outcomes, O/E) and discrimination (such as sensitivity and specificity) of the model.
Similarly, if prognostic models report the incremental value of including tests, we will qualitatively summarise the added value reported in each study (such as net reclassification improvement, change in C statistic, etc.).

Evidence synthesis Diagnostic/predictive utility
Where possible, meta-analysis will be applied to synthesise the 2 × 2 tables from each study and produce summary estimates of test accuracy in terms of sensitivity, specificity, and positive/negative likelihood ratios for each platelet function test (and for each threshold specified if possible). The overall prevalence (absolute risk) of each outcome will also be summarised across studies, alongside the outcome prevalence in each test group separately. Analyses will be considered for a composite endpoint of all vascular events as the primary outcome, and then each type of vascular event individually as secondary outcomes.
A bivariate random effects meta-analysis model will be applied, where possible, using maximum likelihood estimation in STATA and Cochrane Revman (v5) software in order to obtain the aforementioned summary estimates of test accuracy measures. Between-study heterogeneity and correlation in sensitivity and specificity caused by different study threshold choices and other factors are accounted for in this method [22], and the exact binomial distribution of the data correctly modelled in each study [23]. Further, the summary receiver operating characteristic (ROC) curve, positive and negative predictive values for a range of outcome prevalences, and the summary diagnostic odds ratio will also be estimated. In order to give a range for the sensitivity and specificity of each test when applied to a specific clinical setting, 95% prediction intervals will also be calculated.

Prognostic utility
To summarise prognostic ability, where possible a random-effects meta-analysis model will be fitted to the extracted data using method of moments [24] to produce summary estimates of the relative risk and odds ratio (and their associated uncertainty) for each test. Where studies have reported results from a 'time-to -event' analysis to account for censored observations (such as a Cox regression), hazard ratios and their associated uncertainty will rather be extracted and then synthesised [25]. Where possible, each threshold will be summarised separately.
Both unadjusted and adjusted results will be extracted from each study, and a separate meta-analysis conducted for each where possible. Unadjusted results inform the prognostic utility of a test when used in isolation. Adjusted results rather reveal whether a test has prognostic utility over and above other prognostic factors; a true causal factor of poor outcome will retain strong prognostic value even after adjustment, and so this will also help to elicit the genuine clinical value of each test.
By default we choose a random effects meta-analysis model to account for likely between-study heterogeneity. Heterogeneity across studies will be measured statistically (using I 2 and τ 2 ) and examined qualitatively. Following each random-effects meta-analysis, a 95% prediction interval will also be calculated to estimate how the Table 1 A 2x2 table to be extracted from each study for  each binary outcome of interest   Test result  Number of  prognostic effect may vary from the average in different contexts and populations [26,27].
Where it is not appropriate to pool data, study results will simply be tabulated and described separately.

Test for funnel plot asymmetry
For each meta-analysis of prognostic utility containing 10 or more studies, the likelihood of publication bias will be investigated using contour-enhanced funnel plots [28] and appropriate statistical tests (such as the Peters Test [29]) for 'small-study effects'; that is, the tendency for smaller studies to provide more positive findings [30].

Meta-regression and subgroup or sensitivity analyses
For each meta-analysis with sufficient numbers of studies (at least 10), sub-group analyses and/or metaregression will be used to explore whether the following pre-specified variables explain any of the heterogeneity: population parameters, definitions of cardiovascular and/or cerebrovascular risk/ diabetes, aspirin dose, test method(s), adherence to treatment, type of outcome, length of follow up, study quality (risk of bias), and test threshold point employed. The sensitivity of the overall results to the inclusion/exclusion of particular studies will be examined. Where results for multiple thresholds exist for studies of diagnostic/predictive test accuracy, each threshold will first be meta-analysed separately, but then a sensitivity analysis will be undertaken according to the multivariate modelling framework of Riley et al. (personal communication), which analyses all thresholds together.
The key subgroup/sensitivity analysis of interest is for adherence to aspirin treatment. This will be achieved by either: i) combining only those studies that present results for patients who adhered to aspirin treatment; or ii) performing a meta-regression where prognostic or diagnostic utility is regressed against the proportion of patients in each study who adhered to treatment. It is recognised however, that meta-regression has low power and is potentially subject to study-level confounding. The second key subgroup/sensitivity analysis of interest will look at the acuteness of cardiovascular disease. This will be achieved by pooling together studies in patients having had an acute coronary syndrome within 12 months, and those presenting with stable coronary artery disease.

Economic evaluation Systematic review of published cost-effectiveness studies
A systematic review to identify any existing economic studies will be undertaken, including cost analyses, costeffectiveness, cost-utility and cost-benefit studies, and decision model-based analyses. Relevant outcomes will be quality-of-life, costs and incremental cost-effectiveness ratios. Quality assessment will be appropriate to the study design e.g. the Consensus on Health Economic Criteria (CHEC) list [31] for economic evaluations and the Philips checklist [32] for model-based analyses. If available, economic models will be reviewed and utilised where appropriate to inform a model-based economic evaluation.

Health economic modelling
If platelet function testing is demonstrated to have diagnostic/predictive utility (in either the total population or a sub-set), the construction of a cost-effectiveness model will be considered. This will require sufficient evidence of the clinical consequences of testing, compared to no testing, in order to populate the model. Ideally this requires evidence from test-treat trials (specific for a particular test shown to have diagnostic/predictive utility). If such studies exist these will be identified in the systematic reviews above. In the absence of such studies, evidence will have to be obtained from other sources or simulated. The nature of any model will be dependent on the results of the systematic review in terms of the test(s) and population(s) in which diagnostic/predictive utility has been deemed acceptable.
However, in the absence of strong data, a speculative model will be constructed that could be populated with data if future research is conducted. A speculative model will be subject to extensive sensitivity analysis, using a range of hypothetical values in order to demonstrate at which point a test may become cost-effective under a number of test-treat effectiveness assumptions.
The economic evaluation will be carried out from an UK NHS and Personal Social Services (PSS) perspective, to take into account health care costs and longer term social care costs of vascular events. A Markov type, state transition model will be the most appropriate model for this decision problem as it can take into account changes in health states and recurrent events over a long period of time.
The model structure may be informed by existing models which consider antiplatelet therapy for secondary prevention of vascular events. If no such models are identified, models considering other methods of secondary (cardiovascular disease) prevention will be sought.
The model will require data on the prognostic and diagnostic/predictive utility of platelet function tests and subsequent interventions for therapy adjustment, including impact on vascular events, bleeding and mortality. Data will also be required on quality of life and health care costs, in particular the cost of testing, drug costs and acute and long term treatment and care of vascular events and bleeding events. For each important model parameter, a point estimate with a range of possible values (confidence intervals) will be determined and a probability distribution around that point estimate constructed. Base-case evaluation will be based on the most likely estimates. Univariate and probabilistic sensitivity analyses will be conducted to deal with uncertainty in model parameters.

Discussion
It is anticipated that the results of this project will represent a significant step towards informing clinical practice by indicating whether specific platelet function tests have diagnostic/prognostic utility in a defined population and could thus lead to a reasonably reliable prediction of the risk of clinically important events. This in turn would inform any decisions on whether platelet function testing should be routinely used in specific populations in order to establish a diagnosis of 'aspirin resistance' and would further support the investigation of candidate adjustments to therapy to compensate for lack of response to standard aspirin. The strength of 'aspirin resistance' based on each test will be determined by its diagnostic/predictive accuracy, its ability to retain prognostic value even after adjustment for other factors, and it retaining good diagnostic/predictive accuracy both in patients who adhered to treatment and those who did not.
If, however, the existing primary studies identified are essentially exploratory in nature, a prospective protocol driven hypothesis-testing study may be required before a change in clinical practice could be justified [33][34][35][36]. The results of the proposed review would greatly facilitate the design of such a study. Should this review of primary studies provide little evidence that a specific test in any population has predictive utility, this would point to a requirement for further improvement or refinement in testing methods before platelet testing could be recommended.
We do not seek individual participant data in this review [33,37]. The decision to collate individual patient data (IPD) is premature until a systematic review has established the current reported evidence regarding the individual prognostic and diagnostic accuracy of existing tests, as described above. In the future, where multiple tests are of interest in combination, for example in a prognostic model, then collection of IPD may be more important. However, it is first important to establish the diagnostic/predictive/prognostic ability of individual tests, as, if they have high diagnostic/prognostic accuracy in defined populations, then a prognostic model including multiple variables is potentially not necessary. Thus, this systematic review will establish how suitable individual tests are, and identify whether subsequent research using IPD data is suitable.