Which strategies support the effective use of clinical practice guidelines and clinical quality registry data to inform health service delivery? A systematic review

Abstract

Background

Empirical evidence suggests data and insights from the clinical practice guidelines and clinical quality registries are not being fully utilised, leaving health service managers, clinicians and providers without clear guidance on how best to improve healthcare delivery. This lack of uptake of existing research knowledge represents low value to the healthcare system and needs to change.

Methods

Five electronic databases (MEDLINE, Embase, CINAHL, Cochrane Central and Cochrane Database of Systematic Reviews) were systematically searched. Included studies were published between 2000 and 2020 reporting on the attributes, evidence usage and impact of clinical practice guidelines and clinical quality registries on health service delivery.

Results

Twenty-six articles including one randomised controlled trial, eight before-and-after studies, eight case studies/reviews, five surveys and four interview studies, covering a wide range of medical conditions and conducted in the USA, Australia and Europe, were identified. Five complementary strategies were derived to maximise the likelihood of best practice health service delivery: (1) feedback and transparency, (2) intervention sustainability, (3) clinical practice guideline adherence, (4) productive partnerships and (5) whole-of-team approach.

Conclusion

These five strategies, used in context-relevant combinations, are most likely to support the application of existing high-quality data, adding value to health service delivery. The review highlighted the limitations of study design in opportunistic registry studies that do not produce clear, usable evidence to guide changes to health service implementation practices. Recommendations include exploration of innovative methodologies, improved coordination of national registries and the use of incentives to encourage guideline adherence and wider dissemination of strategies used by successful registries.

Background

Clinical practice guidelines (CPGs) have long been used to inform best clinical practice by compiling the latest, high-quality evidence on the diagnosis, treatment and management of serious acute and chronic illnesses. In Australia, CPGs are generally condition specific and written by clinicians and experts, including consumers.

Clinical quality registries (CQRs) are a more recent advancement, systematically monitoring the quality of healthcare by using the collected data to identify benchmarks and variation in health outcomes. Variations are fed back to clinicians to inform clinical practice, thereby creating a feedback loop, as a mechanism for continuous quality improvement [1].

The growth in the development and acceptability of CQRs was underpinned by the Learning Healthcare Systems (LHS) framework, first outlined by the former United States Institute of Medicine (IoM) in 2006 [2]. Inherent in the LHS is the sequential, circular model where research influences practice and practice influences research [3]; the CQR feedback process is an integral element in the LHS framework, enabling the change management domain by supporting the process of continuous quality improvement (QI). The Australian Commission on Safety and Quality in Health Care (ACSQHC) has highlighted the critical function of CQRs to deliver QI to our health services [1, 4,5,6].

CQRs function to inform, monitor and report on the implementation of QI measures, aiming to reduce variation and extend best practice care to as many patients as possible, delivering more equitable outcomes [7]. Registries that collect data without benchmarking it against best practice (either through CPGs or high-quality clinical settings) will not have the knowledge to learn from the data and improve quality outcomes. Data entered into the CQR must be locally relevant with implementation supported by those “who know how to initiate, perform, and evaluate quality improvement, and who have the resources to do so” [8].

The change management or implementation phase of the LHS is often less well managed [3, 9], meaning practice is not informed by the relevant outcome data within a reasonable time frame. There are many possible explanations for this, including segregated and siloed groups responsible for knowledge generation, knowledge translation and patient care, the passive role of health consumers and challenges of maintaining communication across silos [10].

Despite there being an abundance of CPGs and CQRs, empirical evidence suggests they are not being used to their full potential [9, 10]. Questions remain about the exact methods and models successful CQRs use to facilitate LHS activities, leaving health service managers, clinicians and other healthcare providers without clear guidance on how best to utilise data and insights from these sources to improve healthcare delivery. An earlier systematic review examined the impact of CQRs as an intervention for improving health outcomes, with the primary interest being mortality or survival [11]. It suggested three reasons why few CQRs had done this well: conflicting roles — quality improvement versus data storage; conflicting use of registry data — for reporting to government and research bodies, but not for assessing and benchmarking health services performance; and a reluctance from registry investors to formally assess registry impact through expensive rigorous trial design [11].

This systematic review addresses a broader research question: which strategies support the effective use of clinical practice guidelines and clinical quality registry data to identify gaps in best practice care and to inform and improve health service delivery? The deliberate framing of the research question in this way supports a more positive exploratory process, focusing on the benefits of utilising existing research knowledge to promote high value health services practices and policies.

Methods

Literature search

Five electronic bibliographic databases were systematically searched using two separate search processes, between 3 and 30 March 2020: MEDLINE, Embase, CINAHL, Cochrane Central and Cochrane Database of Systematic Reviews (the latter to identify other related reviews). The search strategies included studies published from 1 January 2000 to 30 March 2020 or later because the aim was to inform Australia’s developing health translation services with reference to the contemporary theories of learning healthcare systems. Detailed information on the search strategies is available in Additional file 1. Ethics approval was not required for this systematic review of secondary data.

Selection criteria

Publications were included if published in English, between January 2000 and March 2020, involving participants of any age, and with no restrictions on country of study or type of medical condition. Included publications presented original empirical research on the attributes and/or impact of CPGs/CQRs on health service delivery (positive, neutral or negative). All design types were eligible (including nonexperimental designs documenting registry processes) providing they included specific information on how the evidence/data was used to inform health service delivery. The only exclusion criteria were particular publication types (study protocols, scientific conference abstracts, reviews of previously published original studies, editorials, commentaries, letters and duplicate publications).

Literature selection

KD worked with Mr. Roderick Dyson, a research librarian at The University of Sydney, to develop the search criteria for the bibliographic database searches. Initial search terms were based on the suggested key words and nominated databases proposed by The Australian Health Research Alliance (AHRA) Health System Improvement and Sustainability Working Group. Mr. Dyson and KD met on several occasions to refine the search criteria, and two separate searches were conducted to ensure as many relevant publications as possible were identified. The specific details of these searches are contained in Additional file 1.

All abstracts identified in the searches were exported to an EndNote version 9 library. KD and CF separately reviewed all abstracts (152 in search 1; 494 in search 2). When exclusion criteria were applied and duplicates removed, this left a total of 231 abstracts for further review (as agreed by KD and CF). Of these, 48 underwent full-text review by both KD and CF via hard copy Microsoft Word documents, and an inter-rater reliability statistic was calculated (see Additional file 1). Differences between reviewers were discussed and a consensus list developed. Reference lists of each included article were searched, and citation searching was also conducted to identify other relevant publications. Academic experts in this field were consulted for additional relevant articles. The selection process is outlined in Fig. 1.

Fig. 1

PRISMA diagram [12]

Data on population, intervention, comparator, outcomes and study design (PICOS) is shown in Tables 1 and 2 and included the following: first author, year and location of study (country); population, setting and response rate; intervention/evaluation; comparator; outcome: models/methods used; and study design and time frame. Data abstracted from the publications for the finding tables (Additional files 5 and 6) included the following: first author, year and location of study (country); study objectives, benchmarking; type of guideline; type of registry; healthcare findings; and what works? In addition, CF checked the accuracy of the PICOS tables, and RM checked the formatting of the findings tables; KD is responsible for the content of the findings tables. The quality assessment was undertaken by Ms. Heilok Cheng, a Postgraduate Research Student at The University of Sydney. The accuracy of this assessment was checked by KD.

Table 1 Included CPG studies detailing population, intervention, comparator, outcomes and study design (PICOS)

Table 2 Included CQR studies detailing population, intervention, comparator, outcomes and study design (PICOS)

Identification of relevant information from the grey literature

KD undertook a simultaneous search of relevant grey literature using a list of suggested sources provided in the initial review template and expanded this list using a “snowballing” approach of checking relevant references and citations (see Additional file 2). The starting point for this search was the Australian Health Ministries Advisory Council (AHMAC) and ACSQHC strategy document [1].

Analytic approach

Data were extracted from included studies and tabulated and synthesised using thematic analysis methods. Findings were grouped and summarised using an inductive and iterative approach. A range of overarching themes were generated [38], highlighting strategies used to support the effective use of clinical practice guidelines and clinical quality registry data to inform and improve health service delivery.

Quality assessment

Quality assessment was undertaken using checklists relevant to each included articles’ study design. Seventeen studies were assessed using the Mixed Methods Appraisal Tool (MMAT), a validated critical appraisal tool relevant for qualitative, quantitative RCTs, quantitative non-randomised, quantitative descriptive, and mixed methods study designs [39]. Each study type was assessed using five questions, with three possible answers: yes, no and cannot tell (Additional file 3). This tool does not recommend calculating an overall score or excluding studies with low methodological quality [39]. In total, 9 studies met all five criteria [17, 21, 22, 25, 30,31,32, 37], two met four [26, 29], one met three [24], four met two [16, 18, 23, 34] and one met none [36].

Four studies [7, 13, 19, 27] were assessed using the Quality Improvement Minimum Quality Criteria Set (QI-MQCS, Version 1.0) tool, a valid and reliable critical appraisal instrument applicable to healthcare quality improvement (QI) interventions [40]. It comprises 16 criteria, with responses of either “met” or “not met” [40] (see Additional file 4). Of these four studies, the overall quality was very high: one met 15 of the 16 criteria [19], two met 14 [13, 27] and one met 13 of the 16 criteria [7].

Four articles could not be assessed for quality with existing tools as they documented processes and procedures of effective registries [20, 28, 33, 35]. A further publication, which described decision support mechanisms, could not be assessed for quality as it required specialist IT evaluation [15].

Results

Figure 1 outlines the selection process. One-hundred and fifty-one CPG-related abstracts and 191 CQR-related abstracts, excluding duplicates, were identified. Of these, 21 CPG and 27 CQR abstracts met eligibility criteria and underwent full-text review (inter-rater reliability scores of 83.5% were attained for the guideline abstracts and 79.2% for the CQR abstracts). A second round of eligibility review of full-text articles, their references and citations resulted in 100% agreement between KD and CF (11 CPG articles and 15 CQR articles; n = 26).

Clinical practice guidelines

The 11 CPG articles included only one randomised controlled trial (RCT) [14], four before-and-after studies [13, 16, 18, 21], four case studies [15, 17, 19, 22], one descriptive study [20] and one survey [23]. The lack of RCT design in ten of the 11 studies meant causation could not be determined, and generalisability of their findings to other settings, conditions or health systems was limited. The therapeutic areas covered included the following: hospital-acquired infections following caesarean section, opioid prescription rates, medical oncology practice, breast cancer, prostate cancer, urinary incontinence, implantable orthopaedic and cardiac devices and atrial fibrillation.

Eight of the 11 CPG articles mentioned registry data [13, 16,17,18,19,20, 22, 23] mostly in terms of using that data to assess the impact of the intervention. Five studies were conducted in the United States of America (USA) [16,17,18,19,20] with one each from Sweden [13], the Netherlands [14], Slovenia/Spain [15], Australia [21], Germany [22] and Denmark [23]. Table 1 shows the PICOS data, while study findings are summarised in Additional file 5.

Clinical quality registries

The 15 CQR articles included the following: four before-and-after studies [26,27,28, 36]; four survey studies [24, 29, 31, 34]; four case studies [7, 25, 32, 35]; and three interview studies [30, 33, 37]. Again, the lack of rigorous study design limited the confidence in findings from these studies and their generalisability. The therapeutic areas included the following: six cardiac, three stroke, one cancer, one surgical site infections and four were non-specific. Five articles reported registry processes not restricted to a specific health condition. Five studies each were from Sweden [24, 25, 31, 32, 37] and Australia [7, 13, 26, 27, 36], two from USA [33, 34], one joint study from Denmark/England [30] and one each from Denmark [29] and Norway [35]. PICOS information is available in Table 2, and study findings are summarised in Additional file 6.

Findings

This review identified five strategies that strengthen adherence to CPGs and utilise CQRs for quality assurance and benchmarking to facilitate best practice health service delivery.

Strategy 1: Providing high-quality feedback on data with transparency of findings

One study evaluated the impact of surgeon-specific performance reports on adherence to treatment guidelines for older women with breast cancer [18]. The authors concluded surgeon feedback “was insufficient to alter established practices”, and guideline adherence data may need to be used as a standard quality indicator of clinician practice to encourage improved adherence [18]. A review of the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) database reported structured feedback did not significantly improve communication or QI as a result of high unit performance at baseline, low surgeon participation, scheduling challenges for structured feedback and limited dissemination to the wider surgical team [36]. In contrast, Paxton et al. [20] report that, back in 2012, eight national Kaiser Permanente (KP) implant registries, covering orthopaedic and cardiac devices, were electronically capturing information from administrative and claim databases to supplement the registry databases. All KP registries include patient demographic data (age, sex, body mass index, ethnicity), diagnosis, devices used, enrolment history and surgical outcomes including revision procedures, surgical site infections, other complications and death. This integrated data system allowed KP to communicate quality improvement and benchmarking information throughout their networks. For example, registry findings were communicated to widespread audiences of surgeons, administrators and clinical staff through a range of fora including “chiefs of service and administrator meetings, the internal website, individualised physician practice profiles, site visits, newsletters, e-mails, and presentations at regional or national conferences” [20]. Furthermore, reports could be generated and communicated at the level of medical centres and even individual surgeons. Centre-specific reports were made available on the KP internal website, and surgeons could request their own performance reports via “secure communication” routes. Such targeted reporting and dynamic feedback provide the opportunity for benchmarking across sites, regions and nationally and “yields measurable objective improvements in care”, as evidenced by an overall decrease in the burden of total hip replacement revision since the implementation of the KP Total Joint Replacement Registry in 2001 [20]. The American Society of Thoracic Surgeon’s Adult Cardiac Surgery Registry goes further, providing information for public reports that rank cardiac surgery groups [33].

Strategy 2: Planning for sustainability of successful interventions

Sustainability of QI is another important consideration. In a study of post-discharge stroke care, the authors noted the importance of measuring “long-term sustainability impact” [26]. A comparison of two cardiovascular quality registries in Sweden found activities focusing on adherence to standard care programmes and on increased follow-up of patients seemed to lead to more long-lasting outcome improvements [24]. Similarly, a Swedish study argued that surveillance of infectious conditions should be continuous, based on findings that post-caesarean surgical site infection rates decreased from 17.4 to 2.6% within the first year of the intervention but increased to 6.4% in the second year. Reinforcement of the intervention measures resulted in a further reduction of the infection rate to 1.1% [13]. An Australian study of prostate cancer imaging reported a focused, clinician-centred education programme led to improved guideline adherence at a regional level [21]. However, this effect did not persist once the intervention was complete, raising the importance of monitoring outcomes and reinstating successful interventions as needed [21].

A direct mechanism for sustainable CQRs is the legal mandate to report specific disease conditions, as noted in the USA in relation to cancer reporting [33] and in Sweden [13] and Norway [35] with respect to infected surgical sites. In addition, the role of financial incentives improves sustainability in non-mandatory registries. For example, in the USA, cardiologists are not legally required to submit patient care information to cardiovascular registries; however, registry participation is increasingly becoming a prerequisite of eligibility for bonus programmes, preferred provider network status and reimbursement [33]. These payer incentives provide a financial lever for on-going registry participation which is now relatively high among cardiologists and cardiac surgeons [33].

Guaranteed funding is another major component of sustainability. In the USA, cardiovascular registries have found a variety of ways to maintain financial stability, including subsidies from pharmaceutical and device manufacturers, dues paid by participating providers and revenues from the sale of registry data and from grants/contracts for studies derived from those data [33]. Funding models are evolving, and financial incentives are playing a growing role. Participation in cardiovascular registries is increasingly required as a condition of eligibility for bonus programmes, preferred provider network status, and reimbursement [33]. The Society of Thoracic Surgeons (STS) registries are funded through STS member dues, data sales and research grants rather than industry funds [33]. The US federal government funds state cancer registries, and many of these that make data available for researchers receive grant and/or partnership funds from academic institutions [19, 23].

In Sweden, registry governance and funding are derived from its 21 regional counties and include financial levers, such as incentive payments to healthcare providers, to participate [25]. While the introduction of pay for performance has led to an increased focus on NQRs in Sweden, it has also led to increasing competition between registers [37]. A Danish study on adoption of urinary incontinence guidelines found 35% of GPs received reimbursement during the first year after the introduction of payment for using a voiding diary [23]. Another Danish study reported that nonmonetary incentives, such as improving patient care and raising acknowledgement for cardiac rehabilitation, emerged as less tangible but strong incentives [29]. In contrast, participation in Australian CQRs “is largely an unfunded activity at the health service level, with neither funder nor health service funding being allocated to collection or review of these data” [7]. An Australian study of QI interventions in stroke included an incentive payment to increase access to stroke care, contingent on a clinician’s minimum proportion of data collected within the Australian Stroke Clinical Registry. The specific impact of the financial incentive alone was not clear however, because other elements were included in the intervention programme [27].

While funding is crucial to provide long-term CQR viability, additional spending does not necessarily lead to improved outcomes. When a Swedish heart failure registry and osteoarthritis registry were compared, there was no difference in the number of sustained improvements of the chosen goal indicators despite substantial additional funding to the heart registry [25]. Therefore, national standards, mandated reporting and public financing as evidenced in the US state cancer registries may be a more effective base for sustainable and effective interventions.

Strategy 3: Collecting, monitoring and reporting data on adherence to CPGs

One way to improve the quality of care is by providing evidence-based care aligned with clinical guideline recommendations [32]. Improvement in adherence to guidelines has been achieved by the Swedish government in acute myocardial infarction and diabetes care through support of more than 100 national quality registries and utilising quality improvement collaboratives (QICs) [25]. A different example shows improved adherence to opioid-prescribing guidelines has been achieved in the USA through a single visit with a trained academic detailer [16]. Unfortunately, no data was available on longer-term adherence.

Models of care associated with improved guideline adherence includes the Fox Chase Cancer Center partners community hospital/academic partnership model, which uses a tumour registry for disease-specific data and undertakes regular audits of participating hospitals for adherence to the National Comprehensive Cancer Network Clinical Practice Guidelines in Oncology [19]. The Kaiser Permanente group of registries verify adherence to national practice guidelines, such as indications for implantable cardioverter defibrillators as established by the American Academy of Cardiology [20]. A study of the National Cardiovascular Data Registry (NCDR) suggested regionalisation of specialised services, in which peripheral hospitals refer to regional centres (“hub and spoke”) model, may be associated with improved guideline adherence [17]. A German study reported patients enrolled in cardiac disease management programmes (CHD-DMPs) were more likely to receive guideline care, although CHD-DMP participation itself did not significantly improve survival [22]. Other studies have found surgeon-related factors, such as mapping surgeon networks [17] and individual surgeon characteristics [18], did not influence guideline adherence.

Two studies on guideline adherence stressed the importance of electronic data systems for decision support. The development of timed medical decision support systems can be used to manage new chronic patients, to audit the clinical actions performed in healthcare centres and to analyse the adherence of clinical actions to the standard procedures described in CPGs [15]. A Dutch RCT reported guideline recommendations for treatment of atrial fibrillation were more comprehensively implemented in the intervention group (nurse-led care) than usual care [14]. The authors suggested this was most likely due to the use of an electronic patient record with incorporated dedicated decision support software based on the guidelines that downsize complexity and improve adherence to recommendations. Another possibility the authors acknowledged is that a multidisciplinary approach in which cardiologists and nurses work closely together and are required to justify protocol deviations may preclude treatment decisions that do not comply with guidelines so facilitating QI [14].

Strategy 4: Forming productive partnerships

Several National Cancer Institute (NCI)-designated Comprehensive Cancer Centers in the USA have established academic and clinical partnership models, including the Duke Cancer Network, MD Anderson Cancer Network, the Memorial Sloan Kettering (MSK) Cancer Centre Alliance [34] and Fox Chase Cancer Center Partners Program [19]. These programmes typically provide community hospital partners with access to consultative services, treatment protocols, research and clinical trials. In return, the academic centres obtain enhanced accrual to and more timely completion of clinical trials and the inclusion of more diverse populations in these studies [34]. These partnerships with community health systems facilitate economies of scale, with shifts to population health and value-based reimbursement models, clinical standardisation and increased consumer engagement [34]. MSK Cancer Alliance taps into routine internal data to benchmark practice for each MSK-determined standard of care. Using data from each institution’s tumour registry and billing and medical records, MSK identifies practice variation within the system and uses this information to inform strategies to address gaps [34].

Productive partnerships between professional associations that address a specific health condition are illustrated by the leading cardiovascular societies in the USA. The American College of Cardiology, the STS and the American Heart Association share several collaborative registries in addition to individual registries [33]. These registries use data integrators, establish partnerships between professional societies in registry design and implementation, implement creative mechanisms for registry financing, provide rapid feedback to providers and use consumer-oriented products to share outcome data [33]. The PINNACLE registry (for outpatient cardiology quality improvement) can directly report back measures to the Centres for Medicare and Medicaid Services and to providers at the individual physician or group level and benchmark performance measures to national averages [33].

Strategy 5: Engaging with the whole CQR team

Several studies have reported the low levels and/or infrequent use of registry data to inform clinical practice or continuous QI activities [8, 20, 29,30,31,32,33]. Where registries often involve siloed registry staff, QI initiatives usually involve frontline staff that have varying knowledge and experience in interpreting registry data and incorporating it into multidisciplinary clinical decision-making [8, 32]. Several solutions have been suggested to engage with the QR team. First, the quality measurement efforts need to be adapted to the requirements of end users. Linking registration of quality indicators to use for QI activities increases their routine use, thus making them meaningful tools for clinicians [8]. Second, simplifying the process by tying data entry to existing routines, such as same day entry, and the use of a more user-friendly technology interface, would facilitate registry use [29]. A third potential solution is to focus on one or two key elements that can help registries be better leveraged for improving healthcare performance [33]. For example, the KP implant registries leveraged the integrated healthcare system’s administrative databases and comprehensive health electronic records [20]. KP used risk calculators, based on multivariable analyses of registry data, which allowed patients and surgeons to make clinical decisions at the point of care. In addition, they were able to use administrative data to conduct performance evaluations comparing similar implants with patient and surgical characteristics [20].

A fourth solution centres on a supportive organisational culture. Findings from interviews with registry workers in England and Denmark noted that management support in the data collection and entry phase was crucial. Their findings showed that getting high-quality data into a registry requires a dedicated, sustained effort that involves not only staff but also all stakeholders [29]. Developing a culture of data reporting may facilitate CQR implementation, along with highly engaged teams or individuals, who act as champions for change [29]. The success of the Swedish Riksstroke registry data for QI was strongly related to the buy-in and interest from managers, physicians and nurses in all 72 Swedish stroke units, and that QI initiatives were facilitated by joint ventures between health professionals, managers and policy-makers [31]. Similarly, in Australia, CQR clinician champions were critical in highlighting the benefits of sharing CQR results within the health service [21]. Clinical champions have a role to counter medical doctors’ influence when it presents as a barrier to new initiatives reaching clinical care [37].

The final solution highlights the importance of ongoing communication. In a large metropolitan hospital in Australia involved in 69 data registries, a range of communication strategies were tested to maintain engagement of clinicians in sharing CQR outcomes. Strategies included the establishment of a registry interest group, a calendar of clinical quality registry reports and a reporting guideline, template and dashboard. The reporting tool and dashboard have shown success at the pilot stage [7]. Communication is required not only within organisations but also between different levels of healthcare organisations responsible for delivering improved patient care. Table 3 highlights the complexities of healthcare delivery when different sections of the health system have different roles and responsibilities.

Table 3 Examples of strategy components and levels of responsibility

Discussion

This comprehensive systematic review of CPGs and CQRs has identified five important strategies to facilitate the translation of research evidence and data into practice, across a range of health conditions and countries. This set of strategies, applied in context-relevant combinations, is likely to be the most successful tools for informing health services delivery options and facilitating those options that have the best chance of improving health outcomes across different health settings, conditions and systems.

The defining feature of a CQR is the feedback loop to clinicians which transforms it from a data repository into the change management component of a LHS [1]. Transparent, high-quality and timely feedback allows individual clinicians to assess how their patient outcomes compare with others and to identify gaps in their own practice that need improvement (strategy 1). This view is supported by a Cochrane review of audit and feedback (A&F) mechanisms that reported feedback worked best with under-performing health professionals, and when it was delivered by a supervisor or colleague, was provided more than once, verbally and in writing and included clear targets and an action plan [41, 42]. These mechanisms may also work to improve the use and usability of registry data. For example, the open reporting systems that form the basis of the Kaiser Permanente registry groups have been successful in providing accessible and useful data to administrators and clinicians that has been used for audit and feedback as well as QI and benchmarking [20].

Sustainability strategies (strategy 2) are echoed in the 2017 Cochrane Effective Practice and Organisation of Care (EPOC) systematic review guidelines which specify a minimum of three data collection time points both before and after an intervention to be eligible for inclusion in a Cochrane EPOC systematic review [43].

Adherence to CPGs (strategy 3) is one of two major ways of benchmarking health services around adherence to best evidence practice. Monitoring adherence with evidence-based CPGs is the primary purpose of some registries in the United States of America (USA). There are examples where even this basic use of registry data appears to have increased use of evidence-based therapy, more efficient patient care and shorter hospital stays [44]. A second method of benchmarking is through comparison of current outcomes against best clinical practice and through trialling QI measures that have been successfully implemented in high-performing clinical settings (strategy 4).

Effective CQR outputs, capable of identifying gaps in services and providing reliable data for benchmarking, require sustained effort by the whole team including those involved in CQR design, management, data collection, data entry and data usage (strategy 5). LHSs aim to simplify the link between knowledge generation and evidence implementation and represent the perfect environment for maximising potential benefits from CQRs. In practice, however, there are few examples of where these systems work well [45].

Overall, the quality of included studies was high. Twenty of the 26 publications were able to be assessed using quality criteria; of these, 15 could be considered high quality (eleven scored 4 or 5 out of 5 on one measure (Additional file 3), and four scored 13 or higher out of 16 on another measure (Additional file 4). However, there was only one RCT in this review; the other study designs (before-and-after, case study, interview and survey designs) were not capable of determining causation of any outcomes. If the aim of a study is to determine whether a specific intervention did or did not lead to quality improvement in patient care, then an RCT design is required with individual components randomised separately to avoid confounding of effects.

More efficient usage of CPG evidence and CQR data would be facilitated by improved coordination of national registries and greater dissemination of information on how successful registries operate. For example, the authors and AHRA Working Group are aware of several high-impact registries that were not represented in the peer-reviewed literature. While some information about these registries is available through their respective websites (see Additional file 2), strategies they employ to inform healthcare delivery are not available. An earlier systematic review argued that evidence of CQR impact does exist, but is not available for review because the empiric studies have not been published [11]. This is a general limitation of reviews in this field.

Greater coordination of registries on a national basis would enable newer CQRs to benefit from the methods and models used by their more experienced counterparts. The Australian Commission on Safety and Quality in Health Care is well-placed to oversee the further development of CQRs and is currently working on a revised framework and the development of quality standards for CQR governance, reporting and technical infrastructure arrangements, to minimise duplication and maximise the use of health data for the community [46].

Innovative methodologies such as implementation laboratories, which encourage the partitioning of major research projects into smaller, more manageable parcels and the sharing of knowledge between groups working in the same area, are one example of a promising approach [45]. Rather than a plethora of poorly designed research projects trying to answer difficult questions about what works best, fewer research groups could focus on a smaller part of the puzzle, with a targeted approach using rigorous methodology [45].

There are several other areas of potentially useful research in this field. Quantitative surveys and qualitative studies that explore the reasons for non-publication of successful examples of best practice are needed to determine the extent of under-reporting and propose solutions. Such research could explore the potential role of positive or negative incentives to promote better coordination and publication of research and consider a greater role for patient views on what works best for the service users through the incorporation of patient-reported outcome and experience measures. The use of hybrid study designs that aim to simultaneously consider both the intervention and the implementation strategies used to promote uptake [47, 48] may also be useful.

Strengths/limitations

This review was rigorously conducted and satisfied the PRISMA-S checklist criteria [49] (see Additional file 7). It covered publications reporting guideline and registry use in 10 countries over a 20-year time period and was not limited to specific health conditions. Analysis of findings was descriptive, as determined by the range of study designs. Despite a thorough search strategy, some relevant publications may have been missed. Reference and citation checking, along with consultation with experts in the field, likely reduced this possibility. While individual primary studies in this review contained variable degrees of selection, confounding and reporting bias, the quality assessment of each study was tailored to the different study designs to account for this.

Conclusion

This extensive review of CPG and CQR studies aimed at improving health service delivery identified a set of five key strategies that have been used to facilitate the translation of research evidence/data into practice across a range of health conditions and countries. The use of these strategies, in different combinations according to their potential utility to address issues in different contexts, may provide a helpful framework for considering options demonstrated to have worked in similar contexts. Greater coordination and knowledge sharing, in conjunction with higher-quality research methodology, are also required to improve evidence uptake, health service delivery and patient outcomes.