Improving quality of care for persons with diabetes: an overview of systematic reviews - what does the evidence tell us?

Background Ensuring high quality care for persons with diabetes remains a challenge for healthcare systems globally with consistent evidence of suboptimal care and outcomes. There is increasing interest in quality improvement strategies to improve diabetes management as reflected by a growing number of systematic reviews. These reviews are of varying quality and dispersed across many sources. In this paper, we present an overview of systematic reviews evaluating the impact of interventions to improve the quality of diabetes care. Methods We searched for systematic reviews evaluating the effectiveness of any intervention intended to improve intermediate patient outcomes and process of care measures for patients with any type of diabetes. Two reviewers independently screened search results, appraised each systematic review using AMSTAR and extracted data from high quality reviews (AMSTAR score ≥ 5). Within reviews, we used vote counting by direction of effect to report the number of studies favouring an intervention for each outcome. We produced summaries of results for each intervention category. Results We identified 125 reviews of varying methodological quality and summarised key findings from 50 high quality reviews. We categorised reviews by quality improvement intervention. Eight reviews were broad based (involving a variety of strategies). Other reviews considered: patient education and support (n = 21), telemedicine (n = 10), provider role changes (n = 7), and organisational changes (n = 4). Reviews reported intermediate patient outcomes (e.g. glycaemic control) (n = 49) and process of care outcomes (n = 9). There was evidence of considerable overlap of included studies between reviews. Conclusions There is consistent evidence from high quality systematic reviews that patient education and support, provider role changes, and telemedicine are associated with improvements in glycaemic and vascular risk factor control in patients. There is less evidence about the impact of quality improvement interventions on other key process measures such as screening patients for diabetic complications. This paper provides decision makers with a comprehensive overview of evidence from high quality systematic reviews about the effects of quality improvement interventions on improving diabetes care.


Background
Diabetes is a complex health problem that results in significant morbidity and mortality and health care resource utilisation [1][2][3]. With projected increases in the incidence of diabetes worldwide, health systems continue to focus on improving and optimising diabetes care by influencing patient behaviour and improving efficiency of care [1][2][3]. Yet, providing high quality care for diabetics still remains a challenge for healthcare systems and providers.
Recognising that gaps exist between best and actual care, researchers, people with diabetes, clinicians and decision makers have shown an increasing interest in quality improvement (QI) strategies to improve diabetes management [4]. Quality improvement strategies are 'multidisciplinary, systems-focused, data-driven methods of understanding and improving the efficiency, effectiveness, and reliability of health process and outcomes of care'. QI strategies attempt to 'reduce the difference between health care processes or outcomes observed in practice and those potentially obtainable based on current evidence-based knowledge' [5].
There is substantial evidence and consensus on what constitutes high quality diabetes care [6]. Despite this, suboptimal care and poor patient outcomes continue at the local, national and international level [7]. For example, In Canada, evidence-based clinical practice guidelines for the management of diabetes have been available since 1998 [8]. While these guidelines have contributed to improvements in care [8][9][10], Ontario diabetes patients are still not optimally managed [9]. For example, between 2005 and 2008 the proportion of diabetes patients who had annual eye exams or foot exams remained fairly static at approximately 51% [9]. Further, only 46% of elderly diabetic patients in Ontario filled prescriptions for both ACE inhibitors and statins despite recommendations that most diabetes patients should receive both [8,11,12].
The current interest in QI interventions to improve diabetes care has led to a profusion of primary studies on diabetes that is overwhelming. Whilst systematic reviews partly address this problem of information overload, decision makers often find it difficult to reliably retrieve and keep up to date with the growing volume of published systematic reviews [13]. In addition, available systematic reviews are of variable quality, complexity and length, and are published in a variety of sources [14]. Further synthesis of this evidence is needed to provide reliable and accessible information to clinicians and decision makers. Overviews of systematic reviews are an efficient way to gather the best available evidence in a single source to provide broad, cumulative statements that summarise the current evidence on the effectiveness of interventions. Such overviews are helpful as starting points for decision makers to unpack the evidence towards finding solutions to improving practice and identify areas where new research is needed. This paper reports an overview of systematic reviews of diabetes QI interventions.

Eligibility criteria
We included systematic reviews that evaluated interventions to improve the quality of diabetes care and management in patients of any age, with any type of diabetes, in any setting compared to usual care or other intervention(s). We defined a systematic review as a synthesis of research evidence in which literature searches, inclusion criteria, and critical appraisal methods were explicitly described. We excluded primary studies, and systematic reviews that addressed multiple chronic diseases where it was not possible to isolate the effects of QI interventions on diabetes care.
To be included, systematic reviews had to report effects on at least one intermediate patient outcome, (glycaemic control as measured by glycated haemoglobin (HbA1c) level. vascular risk factor control as measured by high-density lipoprotein, total cholesterol, or blood pressure levels, or maintaining smoking cessation for at least one year) or process of care measure (monitoring HbA1c levels in patients, prescribing appropriate medications such as acetylsalicylic acid, statins, or antihypertensive drugs to control vascular risks, conducting retinopathy screening or referring patients for eye examinations, performing foot examinations to screen patients for potential problems such as ulcers or infections, monitoring renal function through testing of creatinine or microalbumin, or prescribing nicotine replacement therapies to promote smoking cessation). We excluded systematic reviews that only reported changes in knowledge or attitudes towards treatment of diabetes [15].

Search strategy to identify systematic reviews
We searched the following electronic databases without language restrictions: Medline, EMBASE, AARP Ageline, AMED (Allied and Complementary Medicine), and HealthSTAR via OVID, The Cochrane Database of Systematic Reviews and DARE (Database of Abstracts and Reviews) via Wiley, Health Systems Evidence (www. healthsystemsevidence.org/), Rx for Change (www. rxforchange.ca/), and Google.
We used two search strategies to identify potentially eligible systematic reviews. The strategies are based on those used by Shojania [16,17] and the Cochrane Effective Practice and Organisation of Care (EPOC) Group. Full details of the searches and rationale are provided in Additional file 1. Search dates spanned 1976 to April 2011.

Selection
Two reviewers independently screened all titles and abstracts identified by the searches. We retrieved the full papers of citations that passed the initial screening, and two reviewers independently assessed each against the eligibility criteria. Reviewers compared results and resolved any discrepancies through discussion or third party adjudication.

Quality assessment
Two reviewers independently assessed the methodological quality of all reviews that met the eligibility criteria using the 'assessment of multiple systematic reviews' (AMSTAR) checklist [18,19], an 11-item validated measurement tool where reviewers score one point for each criterion met, with higher scores indicating a higher level of methodological quality. Items on the AMSTAR checklist assess criteria such as the comprehensiveness of the search and whether the quality of included studies was evaluated and accounted for [20]. Reviewers compared scores for each item of the AMSTAR checklist and resolved disagreements through discussion or third party adjudication. To present the best available evidence, we extracted data from those systematic reviews that scored five or above [20].

Data extraction
Two reviewers independently extracted data from each included systematic review using standard forms developed for this overview. We used a consensus process to ensure the consistency and reliability of the data and enlisted the assistance of a third party in cases of disagreement. For each included systematic review, we extracted review level information on the objectives, publication year, number of included studies, search dates, country of origin, method(s) of analysis used, as well as relevant quantitative data such as pooled effect sizes. We did not retrieve primary study publications; rather we abstracted primary study data reported within each systematic review on study design, population, intervention(s), comparator, and direction of effect, whether reported descriptively or numerically. We classified the interventions used in each study using the McMaster Health Forum taxonomy [21].

Analysis
At the review level, we categorized each systematic review by quality improvement category (Additional file 2). We analysed, summarised, and reported separately the results of all relevant comparisons within each systematic review using quantitative and qualitative methods as appropriate. We used vote counting as our method of analysis [22]. We counted the number of studies showing a positive direction of effect. If the review reported conflicting results or we were not able to determine the direction of effect, we classified the outcome as unclear. We excluded studies from our analysis that were beyond the scope of this project and reported this for each review.
We considered the results of the intervention effects featured in reviews when five or more of the included studies reported on a given outcome. We determined an intervention to be generally effective when the results of 67% or more of the included studies favoured the intervention; to have mixed effects when the results of 34% to 66% of the included studies favoured the intervention; or to be generally ineffective when the results of fewer than 34% of the included studies favoured the intervention. If there were fewer than five studies reporting on a specific intervention outcome within a systematic review, we considered this to be insufficient evidence to contribute to our overall analysis of that intervention. Figure 1 details the flow of information through the different stages of this overview using the 'preferred reporting items for systematic reviews and meta-analyses' (PRISMA) flow diagram [23]. Our searches resulted in 5,792 article citations. After the initial screening of titles and abstracts, we retrieved 304 articles for full-text review. Of these, we excluded 179 articles that did not meet the eligibility criteria. The remaining 125 systematic reviews were published in 76 journals between 1990 and 2011. We assessed the methodological quality of these systematic reviews and found a median AMSTAR score of 4 (interquartile range, 2 to 6). Fifty systematic reviews were reported in 54 papers (published in 27 different journals), had an AMSTAR score of 5 or more and were included in the overview, and are described in Additional file 3 [16,17,. Details of the 75 excluded reviews are provided in Additional file 4.

Quality of systematic reviews included in the analysis
The majority of reviews met AMSTAR criterion relating to the following: conducting a comprehensive search of the literature (49/50); providing a list of the characteristics of included studies (49/50); using appropriate methods to combine the findings of the included studies (48/50); assessing and documenting the scientific quality of included studies (47/50), and using the results of the scientific quality assessment in formulating conclusions (38/50). However fewer studies met AMSTAR criteria relating to the following: using independent reviewers for selecting studies and extracting data (29/50); using publication status as part of the inclusion criteria (25/50); assessing the likelihood of publication bias (22/50); providing an a priori design (21/50), and providing a list of the included and excluded studies (18/50). Only one review met the criteria for reporting conflict of interest of review and studies ( Table 2).

Synthesis of broad based reviews
We identified eight systematic reviews evaluating the effectiveness of a range of QI interventions directed to one or more of patients, providers, and healthcare systems (Additional file 3) [16,31,43,61,63,66,71,75]. Three reviews only included RCTs with or without CCTs. Patient-directed interventions involved education or information provision; [16,31,43,61,63,71], providertargeted interventions included the use of reminders or prompts, educational materials, meetings, and outreach [16,43,63,66,75]; system-targeted interventions included changes in the physical structure of healthcare facilities, introduction of health records systems or registries, and changes to the site of service delivery [16,63,75]. Three reviews examined the effectiveness of multiple approaches in specific patient populations: older adults [31], socially disadvantaged groups [43], and minority ethnic groups [66]. Among these reviews, we noted that provider role changes such as role expansion and use of multidisciplinary teams [16,31,43,61,63,66,71,75], and telemedicine [16,43,63,75] interventions were frequently evaluated. These reviews observed a number of likely effective interventions. For example, Shojania and colleagues [16,17] reviewed 66 studies (including 50 RCTs) and 11 different QI interventions and observed a mean reduction in HbA1c of 0.42%. Meta-regression identified two interventions that were associated with HbA1c reductions greater than 0.5%, team changes, and case management.

Telemedicine
We identified ten systematic reviews examining the effectiveness of telemedicine technology in the provision of diabetes care to local and remotely based patients (Additional file 3) [28,39,49,56,62,64,67,68,70,73]. These interventions consisted of the transmission of blood glucose values by patients via phone (including mobile or fax) and computer (Internet or website) to healthcare providers for review, with feedback to patients by phone, videoconference, or other electronic means. Seven reviews only included RCTs with or without CCTs. Two reviews addressed interventions targeted only to children and young adults [67,68]. Three reviews described a system interface where data were transmitted to a remote server for analysis, after which appropriate automated messages or reminders were sent to patients or their providers [28,39,70].
All reviews reported on the clinical effectiveness of the interventions on glycaemic control in patients. Telemedicine interventions improved HbA1c levels in eight reviews [28,49,56,64,67,68,70,73], and three reviews had mixed results [39,62,70]. Where reviews examined different modes of data transmission, it was found that SMS (short message systems), when used alone or in conjunction with the Internet to deliver home glucose records and support, were generally associated with improved glycaemic control in patients [49,68]. Internet as a primary means of transmission of blood glucose data and support also had a positive effect on glycaemic control [68].

Provider role changes
Seven systematic reviews examined the effectiveness of changing, expanding, or integrating the roles of healthcare professionals to improve diabetes care and outcomes (Additional file 3) [25,33,50,51,54,57,74]. Three reviews only considered RCTs with or without CCTs. Five systematic reviews examined the impact of role expansion by increasing the responsibilities of pharmacists to include medication management, patient education, and support [50,54,74] and expanding the role of the nurse to include educating and monitoring diabetes patients [33,51]. Other provider role change interventions included role substitution and the use of multidisciplinary healthcare teams. In one of these reviews, local community health workers substituted for medical professionals or worked as part of a multidisciplinary team to provide socioeconomic or culturally appropriate care [57]. In a second review, general practitioners substituted for psychological specialists [25].
All reviews reported on the clinical effectiveness of the interventions on glycaemic control or vascular risk factor control in patients. Interventions that involved changing, expanding, or integrating the roles of healthcare professionals were associated with improvements in patients' glycaemic [25,50,51,54,57,74], cholesterol [54,74], and blood pressure levels [33,54,74].

Organisational changes
Four systematic reviews examined the effectiveness of organisational changes to improve diabetes management and patient care (Additional file 3) [24,32,40,44]. Two reviews only considered RCTs with or without CCTs. Reviews compared the effectiveness of hospital vs homebased [32] specialist or general practice care [44], point of care testing for HbA1c at the time of patient consultation [24], or shared decision making between primary care physicians and specialists in patients receiving ambulatory care [40]. The interventions in these systematic reviews were heterogeneous and included the use of patient/provider reminder systems, protocols, multidisciplinary teams, and interactive communication between primary care physicians and specialists. One review focused solely on outpatient paediatric populations [32]. Changes to patient HbA1c levels were the only outcome reported in these systematic reviews.
The systematic reviews in this intervention category had relatively few included studies and thus provided insufficient evidence to allow us to determine if the organisational changes were associated with improved glycaemic control in patients. Only one systematic review included sufficient studies showing that shared decision making between primary care physicians and specialists improved blood glucose levels [40].

Summary of the evidence
In this overview, we identified 125 systematic reviews that evaluated the effectiveness of quality improvement interventions to improve diabetes care. We excluded 75 of these from further assessment due to low methodological quality and undertook a detailed analysis of 50 high quality reviews. The majority of included reviews only considered RCTs with or without CCTs; in the remaining reviews that included other designs, RCTs and CCTs were the commonest designs included. The eight broad based reviews examined a range of different interventions that led to improvements in patient self-management outcomes (glycaemic control and cholesterol levels) and process of care behaviours (HbA1c and retinopathy monitoring) with mixed results for blood pressure control. Based on our assessment of 42 high quality intervention specific reviews, patient education and support interventions were shown to improve HbA1c, blood pressure, cholesterol, and diabetic foot outcomes in patients; telemedicine interventions were associated with improved glycaemic control in patients, and provider role change interventions improved glycaemic and vascular risk factor control in patients. It was unclear what impact organisational interventions had on glycaemic control in patients. It was also unclear if there was a relationship between the above four interventions and improvements in monitoring of HbA1c, vascular risk factors, or retinopathy, or diabetic foot outcomes. The majority of reviews only included randomised trials alone (with or without controlled clinical trials). In general, the results and conclusions of the systematic reviews that only included RCTs with or without CCTs were similar to those including a broader range of designs. We were unable to identify any high quality reviews that focussed on other QI interventions relevant to our objectives, such as financial or regulatory interventions.

Strengths and weaknesses of the overview
Our objective was to synthesise a comprehensive body of published evidence in a single overview. This is a challenging undertaking considering that overview methods are still evolving and a variety of approaches are being used [20,76,77]. Strengths of our approach include the use of explicit methods to identify, appraise and summarise available systematic reviews of interventions to improve diabetes management and outcomes. We employed sensitive search strategies that were developed and run by an information specialist with expertise in searching for interventions to improve health care delivery and healthcare systems. Two authors independently undertook study selection, quality appraisal using the validated AMSTAR tool, and data abstraction with a consensus process to address disagreements. Further, we performed a reanalysis of the results of reviews using vote-counting to ensure consistency of analytical approach when considering results across the reviews. We focussed on high quality reviews (any review scoring ≥5 on AMSTAR) as our previous experience suggested that Table 3 Summary of results from included reviews on outcomes (Continued)

Organisational changes
Al-Ansary 2011 [24] 7 0 reviews with lower AMSTAR scores are difficult to interpret and likely do not provide reliable evidence. Our approach allowed us to explore whether different review teams addressing similar review questions independently observe similar results and draw broadly similar conclusions. The consistency of findings and conclusions across high quality reviews, for example that patient education and support improves many aspects of diabetes care, suggests that this is not likely a spurious finding due to the review team or methods chosen. However, overviews inevitably suffer from potential weaknesses, some of which are common to all synthesis projects and others that are specific to overviews. Potential weaknesses that are common to all syntheses include the possibility that our searches missed relevant reviews, or that errors were made during study selection, quality appraisal and data collection. Overview-specific weaknesses relate to the fact that our unit of study is a completed systematic review. Thus, overview authors are dependent upon the methods of the included systematic reviews that potentially suffer from some of the common weaknesses mentioned above. Overviews that consider multiple reviews on the same topic potentially protect against weaknesses in individual reviews unless all included reviews share precisely the same methodological weakness. In our overview we also tried to minimise the likelihood that we would be misled by individual reviews by excluding low quality reviews that are more likely to suffer to from major methodological weaknesses.
Overviews are also dependent on the reporting of the included systematic reviews, which limits the granularity of detail available to the overview author. As a result, the overview author needs to trust the systematic review authors' quality appraisal and data abstraction and is limited by the level of detail in the original systematic reviews when describing characteristics of included studies such as setting, and intervention, etcetera. As well, overviews are limited by the coverage of the identified systematic reviews. For instance, there has been increasing interest in the use of financial interventions to improve diabetes care; the United Kingdom introduced a pay-for-performance incentive program in 2004, rewarding family practices for achieving performance targets in chronic disease management, including diabetes [78]. However, despite this policy interest and the availability of primary studies, we were unable to find any high quality systematic reviews that addressed the effects of financial interventions on quality of diabetes care. Finally, overview authors should expect considerable overlap in the primary studies summarised in the included systematic reviews. As a result it is important not to treat systematic reviews as independent observations but rather see included systematic reviews as a different lens addressing the same question to determine whether different teams draw broadly similar conclusions.
The interventions examined in these reviews were frequently complex. Reporting of complex interventions is often poor in primary studies [79] and even poorer in systematic reviews that may reduce a short description in a primary study to a few words in a table. This creates a number of difficulties for overview authors; whilst overview authors might be confident that a review does address the overview question frequently, there will be insufficient detail within the review to be able to describe the evaluated reviews in detail, or to determine effective components of complex interventions, or explore potential effect modifiers.
Within our overview, we reanalysed the included reviews using vote counting based on direction of effect. The weaknesses of vote counting are well documented and include the failure to provide an estimate of the effect size of an intervention, failure to take into account the precision of the estimate from the primary comparisons, and giving equal weight to comparisons with 100 or 1,000 participants [22]. Nevertheless, we would argue that vote counting provided a flexible approach to consider effectiveness across reviews with few assumptions given that the included reviews involved different study designs, presented individual study reviews using different metrics and used a variety of analytical approaches (including descriptive analysis, vote-counting, metaanalysis and meta-regression). To increase the confidence, we reported separately, the number of RCTs that contributed to the overall effect for each intervention comparison. Further, we also reported the results of any meta-analyses performed in the reviews that had bearing on the interventions and outcomes of interest.

Implications for policy
Overviews are high-level syntheses of research evidence that provide an evidence map for decision makers and high-level conclusions about an issue. However, it is likely that decision makers will also need to consult some of the included systematic reviews, and potentially the individual studies, to address their specific questions. Thus, this synthesis is best seen as an entry point to evidence to inform healthcare decision makers' policy options about interventions to improve diabetes outcomes. Current initiatives to improve diabetes management and care should be informed by this evidence base. There is consistent evidence that a number of interventions, for example, patient education and support, telemedicine, and provider role changes, appear to improve diabetes quality of care, and policy-makers need to consider whether their current services optimally provide these effective interventions to their population of persons with diabetes. Overall, there is some evidence to suggest that tailoring interventions to specific cultural or age groups may be beneficial to the effectiveness of the treatment strategy. Interventions at an organisational level had insufficient evidence from which to draw conclusions about their effectiveness, and therefore policy makers should be cautious about implementing these approaches until further evidence is available.

Implications for research
There is a considerable body of evidence evaluating interventions to improve diabetes care that should inform future research. Researchers conducting both primary studies and systematic reviews of diabetes quality improvement interventions should consider the breadth of outcomes relevant to excellent diabetes care. In areas with a substantial number of trials demonstrating benefit, for instance, patient education and support, future research should focus on direct comparisons of different delivery methods for the same intervention or of direct comparisons of the relative effectiveness of different interventions.
The results of available systematic reviews should inform the choice and design of evaluated interventions and evaluative methods. For example, new systematic reviews of diabetes QI interventions might focus on interventions where there are few or no current systematic reviews. Patient education provides an example of a group of interventions where the depth and quality of the evidence is abundant and where further reviews may add little to our knowledge unless they address secondary questions such as how to optimise or extend the reach of patient education and support interventions. While we identified some evidence of attempts by researchers to separate the effects of complex interventions, future studies should further focus on such separation in order to provide specific advice on how to optimise efforts to improve diabetes management. Given our observation that 60% of the reviews assessed for this project were rejected based on low AMSTAR scores, researchers are strongly encouraged to consider AMSTAR assessment criteria during the systematic reviewing process in order to improve the methodological quality and/or reporting of their work.

Conclusions
Overviews provide high-level summaries of empirical research. This overview identified and summarised the best current available evidence from 125 systematic reviews on the effectiveness of different QI interventions to improve diabetes care. The results suggest that patient education and support, provider role changes and telemedicine are associated with improved patient outcomes. They can be used by decision makers to identify policy options to improve diabetes care and as a source document to identify systematic reviews and individual studies that are relevant to their context and that address their specific questions. It also identified potential areas for future research; highlighting the problem of (likely) inappropriate duplication of effort between existing systematic reviews and the lack of high-quality systematic reviews addressing interventions of policy interest, such as regulatory or financial interventions.