Systematic review and meta-analysis of diagnostic accuracy of detection of any level of diabetic retinopathy using digital retinal imaging
Systematic Reviews volume 7, Article number: 182 (2018)
Visual impairment from diabetic retinopathy (DR) is an increasing global public health concern, which is preventable with screening and early treatment. Digital retinal imaging has become a preferred choice as it enables higher coverage of screening. The aim of this review is to evaluate how different characteristics of the DR screening (DRS) test impact on diagnostic test accuracy (DTA) and its relevance to a low-income setting.
We conducted a systematic literature search to identify clinic-based studies on DRS using digital retinal imaging of people with DM (PwDM). Summary estimates of different sub-groups were calculated using DTA values weighted according to the sample size. The DTA of each screening method was derived after exclusion of ungradable images and considering the eye as the unit of analysis. The meta-analysis included studies which measured DTA of detecting any level of DR. We also examined the effect on detection from using different combinations of retinal fields, pupil status, index test graders and setting.
Six thousand six hundred forty-six titles and abstracts were retrieved, and data were extracted from 122 potentially eligible full reports. Twenty-six studies were included in the review, and 21 studies, mostly from high-income settings (18/21, 85.7%), were included in the meta-analysis. The highest sensitivity was observed in the mydriatic greater than two field strategy (92%, 95% CI 90–94%). The highest specificity was observed in greater than two field methods (94%, 95% CI 93–96%) where mydriasis did not affect specificity. Overall, there was no difference in sensitivity between non-mydriatic and mydriatic methods (86%, 95% CI 85–87) after exclusion of ungradable images. The highest DTA (sensitivity 90%, 95% CI 88–91%; specificity 95%, 95% CI 94–96%) was observed when screening was delivered at secondary/tertiary level clinics.
Non-mydriatic two-field strategy could be a more pragmatic approach in starting DRS programmes for facility-based PwDM in low-income settings, with dilatation of the pupils of those who have ungradable images. There was insufficient evidence in primary studies to draw firm conclusions on how graders’ background influences DTA. Conducting more context-specific DRS validation studies in low-income and non-ophthalmic settings can be recommended.
Diabetes mellitus (DM) is one of the most prevalent non-communicable diseases and has significant impacts on health systems . The International Diabetes Federation (IDF) estimated that there were 425 million people with DM (PwDM) in the world in 2017 which is projected to increase to 629 million by 2045 . The greatest impact affects low- and middle-income countries (LMIC) (overall increase 69%) due to ageing population, obesity and sedentary life style . This is exacerbated by weak health systems coupled with slow economic development . Diabetic retinopathy (DR) is a common microvascular complication of DM caused by chronic hyperglycaemia . A pooled meta-analysis using population-based studies conducted in the USA, Australia, Europe and Asia showed that the prevalence of any DR in PwDM aged 20 to 70 years was 34.6% (95% CI 34.5–34.8%): proliferative DR affected 6.96% (95% CI 6.87–7.04%) and sight-threatening DR (STDR) affected 10.2% (95% CI 10.1–10.3%), globally translating to approximately 28 million PwDM affected by STDR . DR is a leading cause of blindness among the young and middle-aged adults in most of the high-income countries (HIC).
Many studies have shown that control of risk factors, early DR screening (DRS) and appropriate treatment can reduce the risk of blindness and visual impairment due to DR [7,8,9,10,11,12]. Digital retinal imaging has been widely practiced and an accurate method for DRS . Providing appropriate training to photographers is of paramount importance, and with enough practice, high levels of competence can be achieved by those taking imaging regularly. Non-mydriatic digital imaging methods cause less discomfort and are more convenient for service providers. However, poor image quality is an important limitation of digital retinal imaging, particularly if non-mydriatic systems are being used, in countries where cataract is common .
In current literature, a systematic review showed that dilated imaging aided by fundoscopy for ungradable images was an effective modality to screen for DR . This review included studies from 1985 to 1998 when digital retinal imaging technology was not available. Shi et al. concluded that accuracy of detecting presence/absence of DR by tele-medicine using digital imaging is high (pooled sensitivity 80%, 95% CI 84–88%; pooled specificity 89%, 95% CI 88–91%) . Another meta-analysis concluded that dilatation of the pupils did not have a bearing on the diagnostic test accuracy (DTA) for any level of DR (sensitivity: odds ratio (OR) − 0.89, 95% CI 0.56–1.41, p = 0.61; specificity: OR 0.94, 95% CI 0.57–1.54, p = 0.80) . A limitation of this review was that results from different imaging methods (i.e. polaroid, film and digital) and clinical examination were pooled into one estimate.
A DRS modality which is suited to the health system and its context is a key factor in the success of a programme . A screening programme requires substantial investment in infrastructure and workforce development. LMICs have low capacity to implement a population-based DRS programme (DRSP) with routine call/recall and full DR patient list. Yet there is a high burden of unmet need, with higher levels of uncontrolled DM leading to higher rates of DR progression. Weak health systems require a DRSP where detection of any DR using most effective and efficient instruments would be most useful. In addition, resources are scarce, and so efficient use of both equipment and human resources are essential. The detection of clinic-based PwDM with any DR will enable identification and stratifying risk groups early and screen safely at a lower threshold at non-ophthalmic settings. Therefore, a feasible way of providing accessible services is to offer digital photographic DRS when PwDM present for routine medical care at diabetologist/physicians’ clinics. In a low-income setting, identification of a person with any DR/no DR would be a helpful stratification for the providers. In a practical programme guideline, we would suggest performing mydriatic imaging or refer to the next level for those with ungradable images. There is also a lack of understanding among the PwDM about the benefits of mydriasis. Discomfort experienced after pupil dilatation has led to low uptake in dilated examination . Therefore, it is important to understand the best method to detect any DR in non-specialist settings that will be suited to LMICs .
The objectives of this review were to evaluate how using or not using pharmacological dilation of the pupil and the number of fields captured influence DTA and how well different ophthalmic and non-ophthalmologist health care professionals perform DR grading compared to seven-field image grading or mydriatic ophthalmoscopy by ophthalmologists in different clinical settings. This will inform decision-making for choosing strategy in those aspects of a DRSP. This is an assessment of accuracy of instruments for a systematic clinic-based screening rather than a population-based screening tool. We plan to propose most efficient modality for provision of DRS to PwDM at non-ophthalmic settings (i.e., medical clinic, endocrinology clinic) using this evidence.
The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines were followed in reporting (The PRISMA checklist is available as Additional file 1).
Eligibility criteria and study context
We included studies of cross-sectional study designs that aimed to evaluate the accuracy of DRS using digital imaging as the index test, in PwDM at permanent healthcare facilities. We used the Early Treatment Diabetic Retinopathy Study (ETDRS) seven-field image interpretation as the gold standard and mydriatic bio-microscopy/ophthalmoscopy by an ophthalmologist/retinologist as the clinical reference standard where the gold standard was not performed. The primary context considered for this review was institutional DRS clinics/programmes using digital imaging. We categorised the context as either primary or secondary/tertiary. We excluded studies conducted in informal health facilities, used automated analysis systems, used non-digital imaging methods in index test, used mobile screening methods or did not report on DTA as an outcome measure.
The outcome examined was sensitivity and specificity of detection of ‘any level of DR’. It is important to understand the optimal method to detect any DR in non-specialist settings, especially in LMICs where PwDM have higher risk of progression, due to poorly controlled risk factors and irregular follow up. ‘Any level’ of DR was considered appropriate as we felt that such an approach would have collateral benefits like raising awareness among the providers as well as augmenting awareness of PwDM regarding the importance of regular follow-up and control of the risk factors minimising the progression to STDR.
Search and study selection
We developed a comprehensive search strategy to obtain published articles by consulting an information specialist and searched MEDLINE (Ovid), Cochrane Database of Systematic reviews (CDSR) and CENTRAL in the Cochrane Library. The databases were searched from the date of inception of the databases to September 2016, to identify any published reviews on this topic and to see whether relevant trials where included in the CENTRAL database. The search terms and strategy are shown in Table 1 and Fig. 1 respectively. Two reviewers (PN and SK) independently assessed the eligibility of the titles and abstracts, and discrepancies were solved by consulting a third reviewer (GV). Full papers of the eligible articles (n = 122) were obtained from the publishers/authors.
Data collection process
A data extraction form was prepared, and data were extracted and entered into a formatted MS Excel® database. Data from all the full reports of filtered citations (n = 122) were extracted. We used a modified Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for cross-sectional studies to identify the components to extract . The modifications made were based on Cochrane guidelines on conducting systematic reviews of studies of DTA . Two independent reviewers extracted the data (PN and SK) from full reports. In the piloting stage, data were extracted from 10% (12/122) of the articles by two reviewers and consistency was checked (SH). Corrections to the data extraction sheets and databases were done at this stage. The data extracted of all the included articles (n = 26) were checked by the co-reviewer (SK) for consistency.
The data extracted from each study included country, study design, study setting, sample size and participant characteristics (mean age with standard deviation and range, male to female ratio, number of years with DM). The next section of the extraction included study objectives, sampling strategy, methods of index test (degree of view, number of fields, pupil status and type of camera) and method of reference standard. Finally, data on DTA (sensitivity with 95% CI, specificity with 95% CI, number of true positives, true negatives, false positives and false negatives, kappa value and gradability) were extracted. Studies were categorised according to the status of pupils, number of fields in imaging, type of index test grader and type of reference standard.
Meta-analysis of the data was conducted to examine differences in outcome due to pupil status (mydriatic and non-mydriatic), number of retinal fields (one field, two fields, greater than two fields), type of index test grader (ophthalmologist, retinologist, retinal reader, ophthalmic registrars) and by the context (primary and secondary/tertiary). A sub-group meta-analysis was undertaken to determine the DTA of ‘any level’ of DR by non-ophthalmic personnel. Further sub-analyses were conducted by considering the studies which reported on DTA using the same participant imaged before and after pupil dilatation.
Risk of bias in individual studies
We assessed the variations in bias using the Quality assessment of diagnostic accuracy studies - 2nd version (QUADAS-2) framework . The methodological quality and applicability of the studies was considered using signalling questions under the four domains of patient selection, index test, reference standard and flow and timing . We examined the differences in reported DTA estimates based on QUADAS-2 quality assessment guidelines, and given results in the meta-analysis were based on the studies identified to have low risk of bias. The methodological quality of the studies included in the review and meta-analysis are described in Table 2. All included studies were cross sectional in design as these demonstrated less bias in the QUADAS assessment. We considered the signalling questions according to the QUADAS-2 guidelines as examples, masking of the graders, inclusion of range of spectrum to reduce the spectrum bias, all participant undertaking all tests etc. when assessing the bias.
Synthesis of results
Meta-analysis was conducted using STATA/IC (version-14.1, 2015-Texas-77845-USA) after acquiring the 2 × 2 table (TP, FP, TN and FN) values for number of eyes screened as the unit of analysis in each method of DRS. These values were cross checked by the number of DR positives and negatives reported in classification of findings under different categories of DR. The meta-analysis was conducted using the DTA of any DR, after excluding the ungradable images. Sub-analyses were conducted using the estimates that reported DTA on same participant groups before and after pupil dilatation and by non-ophthalmic index test graders.
Heterogeneity was assessed between the studies and between different modalities in the same study. Due to differences in definitions of the ungradable image category, we decided to exclude all ungradable images to minimise heterogeneity. At a practical programme level, all PwDM with ungradable images will be referred to the ophthalmologist’s clinic for further assessment. However, in this study, we were interested in the accuracy of the intervention to detect any DR, rather than any referable PwDM in a programme model.
The electronic database search yielded 6646 titles and abstracts, and 122 studies were selected to review full reports. Twenty-six studies were included in the review (Fig. 1). The details of the excluded articles are available as Additional file 2. We included 26 cross-sectional studies, and 88% (23/26) were conducted in HICs [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45]. The remaining studies (3/26, 11%) were conducted in South East Asian upper middle-income countries (Thailand (one) , China (one)  and Taiwan (one) ). There were 6 studies (10 estimates) which reported DTA in which the same participant underwent imaging before and after pupil dilatation [25, 35, 40, 42, 44, 47].
The mean sample size of the studies was 316 PwDM screened (SE± 72.3, 95% CI 166–467, range 51–1549). Thirty percent (8/26) of studies selected participants from local and regional primary care units. Other studies recruited PwDM from retinal care (5/26, 19.2%), diabetes care (4/26, 15.3%), existing DR screening programmes (4/26, 15.3%), medical and ophthalmology care (1/26, 3.8%), retinal and ophthalmology care (1/26, 3.8%), ophthalmology care (1/26, 3.8%) and private sector optometry network (1/26, 3.8%). One study did not report the setting (1/26, 3.8%). The mean age of participants was 57.4 years (SE± .52, 95% CI 54.3–60.7, range 16–89 years): the mean age of participants in non-mydriatic strategies 58.9 years and mydriatic 59.0 years. The mean duration of known diabetes among participants was 12.0 years (SE ± 1.5, 95% CI 8.8–15.3 years), and 50.5% were male (SE ± 2.7, 95% CI 44.8–56.3). Participants’ characteristic tables of the studies included in this review are available as Additional file 3.
Of these 26 studies, 5 studies (5/26, 19.2%) were not eligible for the meta-analysis. Those were excluded from the meta-analysis due to unavailability of required 2 × 2 table data, very high level of bias and heterogeneity. The study conducted by Perrier et al. used the same participants as in the study by Boucher et al. which has been included and another study was excluded due to a high likelihood of bias [33, 36]. The study conducted by Schiffman et al. was excluded as index test pupil status and number of retinal fields were not mentioned . Two further studies were excluded: one only reported DTA for STDR  and another from Singapore (Bhargava et al.) did not provide DTA data .
Among 21 studies included in the meta-analysis, 39 different modalities were identified in terms of pupil status, retinal field strategy and human resources involved in index test DR grading. Forty-six percent (18/39 modalities) of the studies used non-mydriatic methods (13/21 studies) [25, 26, 29, 31, 35, 38, 40, 42, 44, 45, 47,48,49], 44% (17/39 modalities) used mydriatic methods (11/21 studies) [23,24,25, 32, 35, 37, 39, 40, 42, 44, 47] and ophthalmic personnel currently trained and practiced in DR grading had performed index test grading in these studies. In 10%, 4/21 [27, 28, 46, 48] newer non-ophthalmologist personnel had performed index test grading. Six studies reported mydriatic and non-mydriatic methods (6/21) [25, 35, 40, 42, 44, 47]. One study reported DTA values for ophthalmic and non-ophthalmic personnel . The DTA of each screening strategy is available in Additional file 4.
Studies included in secondary output analysis
Four studies were eligible for secondary output of meta-analysis of DTA of DRS as they used different non-ophthalmologist personnel [27, 28, 46, 48]. However, there were no adequate number of studies to meta-analyse by pupil status and field strategy. The details of these studies are described in Additional file 3 (participants’ characteristics) and Additional file 4 (DTA).
Risk of bias and applicability concerns within studies
The methodological quality and applicability assessment of the included studies (Table 2) were according to the QUADAS-version 2 guidelines. In the assessment of bias, it was minimal (15.38% high risk) in conducting the index tests and reference tests. Nineteen percent of the studies showed high risk of bias in selection and 30.7% in participant flow and timing (Fig. 2). In the assessment of applicability, risk was minimal in reference standard (3.8%) and 34% of the studies showed high risk in applicability with regard to patient selection and 50% in index test (Fig. 3).
Risk of bias in the included studies
There was selection bias in some studies: Baeza et al. excluded patients who had visited an ophthalmologist within 6 months of screening and those with hyper-mature cataract  and Boucher et al. purposively selected participants who had a greater risk DR . There were also applicability concerns when authors reported the DTA of referable level of DR [38,39,40, 43, 47]. The study conducted by Hansen et al., which selected people with diabetes through a record review, was weighted towards less severe retinopathy, as mentioned by the authors . Two studies attempted non-mydriatic methods and ended up dilating the pupils due to high proportion of ungradable images [23, 32]. In the study by Lopez-Bastida et al., the time interval between the index and reference tests was not stated, nor whether participants with ungradable images (90/773, 10%) underwent mydriasis while performing the index test . Similarly, time and flow was not mentioned in the study by Ku et al. . Two studies selected indigenous populations which lead to generalizability concerns [32, 37]. Furthermore, some studies were conducted in eye/retinal clinics where there was a possibility of high prevalence of advanced DR [39, 43, 48].
Reporting of DR was not uniform. In several studies, DTAs were reported for different levels of DR leading to some heterogeneity [25, 26, 31, 38,39,40, 43]. In these studies, we considered results for the detection of any level of DR. For example, Phiri et al. had defined DR including the macular signs which other authors had not considered and which would have an impact on the analysis .
Diagnostic test accuracy in non-mydriatic imaging
Among the 21 studies included in the meta-analysis, 18 used the following non-mydriatic imaging strategies: one field (8/18, 44.4%), two fields (4/18, 22.2%) and greater than two fields (6/18, 22.2%). The pooled sensitivity of detection of any level of DR using non-mydriatic digital imaging was 86% (95% CI 85–87%). The two-field strategy gave the highest estimate of sensitivity of 91% (95% CI 90–93%). The one and greater than two field strategies gave summary estimates of sensitivity of 78% (95% CI 76–80%) and 88% (95% CI 86–91%), respectively (Fig. 4, Table 3). The mean proportion of ungradable images in non-mydriatic methods was 18.4% (SE ± 2.2, 95% CI 13.6–23.3%). The summary estimate of specificity of detection of any level of DR using non-mydriatic digital imaging was highest in the two-field and greater than two field strategies (94%, two field 95% CI 93–95%, greater than two field 95% CI 93–96%). The one-field strategy gave pooled specificity values of 91% (95% CI 90–92%) (Fig. 5 and Table 3).
Diagnostic test accuracy in mydriatic imaging
The highest pooled sensitivity of detection of any level of DR using different mydriatic digital imaging field strategies was for the greater than two field strategy (92%, 95% CI 90–94%). The sensitivity of the one-field strategy was 80% (95% CI 77–82%), and it was 85% (95% CI 84–87%) for the two-field strategy (Fig. 6, Table 3). The mean proportion of ungradable images for the mydriatic method was 6.2% (SE± 2.2, 95% CI 1.7–10.8%). The summary estimation of specificity in detection of any level of DR using mydriatic digital imaging was highest in the greater than two field strategy at 94% (95% CI 93–96%) followed by the one field, 93% (95% CI 92–94%) and then two field 82% (95% CI 81–83%) (Fig. 7, Table 3).
Hierarchical summary receiver operating characteristics (HSROC) curve interpretation
Both non-mydriatic and mydriatic strategies showed very high discriminative power in ruling out the presence or absence of any level of DR with the diagnostic odds ratio (DOR) of non-mydriatic strategies being 68.03 (95% CI 35.5–130.0) and positive likelihood ratio of 11.79 (SE 3.04, 95% CI 7.1–19.5) (Fig. 8). Similarly, mydriatic DOR was 53.98 (95% CI 31.1–93.5) and the positive likelihood ratio was 9.5 (SE 2.1, 95% CI 6.1–14.7) (Fig. 9). After adjusting for ungradable images, we observed that the pooled sensitivity of detection of any level of DR was the same for non-mydriatic and mydriatic strategies: 86% (95% CI 85–87%) for both. The specificity of detection of any level of DR was higher using both non-mydriatic and mydriatic greater than two field strategies (94%, 95% CI 93–96%) and in two-field non-mydriatic strategy (94%, 95% CI 93–95%). The highest DOR was obtained for the greater than two field strategy (non-mydriatic DOR 182.4 (SE 145.2, 95% CI 38.3–868.5), mydriatic DOR 140 (SE 76.1, 95% CI 48.2–406.7)). Therefore, we have to consider the number of fields in a DRS strategy (Fig. 10 and Additional file 5).
Summary estimates were derived by the reference test, to assess the variability in DTA according to the reference standard. The pooled sensitivity of detection of any level of DR was higher in non-mydriatic imaging using seven-field ETDRS images as the reference than direct/indirect ophthalmoscopy: (87%, 95% CI 85–89% vs 86%, 95% CI 85–88% in mydriatic). There was no significant difference when compared with mydriatic bio-microscopic ophthalmoscopy as the reference standard (non-mydriatic 86%, 95% CI 85–88% vs mydriatic 86%, 95% CI 85–87%). Pooled estimates of specificity were high in both non-mydriatic (96%, 95% CI 95–97%) and mydriatic (96%, 95% CI 95–97%) imaging using seven-field ETDRS images as the reference standard compared to mydriatic bio-microscopy (non-mydriatic 91%, 95% CI 91–92 vs mydriatic 87%, 95% CI 86–88%) (Table 3 and forest plots available in Additional file 6).
In the analysis of DTA by setting, the highest estimates were shown in secondary/tertiary settings using non-mydriatic imaging (sensitivity 90%, 95% CI 88–91; specificity 95%, 95% CI 94–96%) compared to mydriatic imaging (sensitivity 87%, 95% CI 86–89%; specificity 89%, 95% CI 88–90%) (Table 4). However, in non-mydriatic methods, there was one study from HIC with a larger sample size, which may have attributed for a skewed result (40).
Regarding the personnel involved in index test grading, for ‘any level’ of DR, the highest pooled sensitivity and specificity using non-mydriatic imaging was reported by retinologists: sensitivity 90% (95% CI 89–92%) and specificity 94% (95% CI 93–95%). The highest DTA estimates in mydriatic imaging were reported by ophthalmologists (sensitivity 87%, 95% CI 85–89%; specificity 93%, 95% CI 92–94%) (Table 4 and forest plots available in Additional file 7).
In the sub-analysis of those studies that captured images of the same participant before and after pupil dilatation, mydriasis (one field, two fields, three fields and five field: six studies, ten estimates) showed a high level of sensitivity: mydriatic 88% (95% CI 86–89) and non-mydriatic 82% (95% CI 80–84%). However, a higher level of specificity was shown in non-mydriatic methods in detecting any level of DR: non-mydriatic 92% (95% CI 91–93%) and mydriatic 89% (95% CI 88–90%). Forest plots of these estimates are available in Additional file 8. Four studies used non-ophthalmologist personnel as primary graders in the index test. The pooled sensitivity and specificity of detecting any level of DR (either non-mydriatic or mydriatic) were 74% (95% CI 71–77%) and 85% (95% CI 83–87%) respectively (27, 28, 46, 48).
Overall, both mydriatic and non-mydriatic digital imaging methods generate a satisfactory level of sensitivity, i.e. 86% (95% CI 85–87%) in usual clinical settings, once ungradable images are excluded from analysis. This sensitivity level is above the DRS recommendation of established national programmes (sensitivity > 80%) . Neither strategies achieved the recommended level of 95% specificity for any level of DR: non-mydriatic 95% CI (92–93%) and mydriatic 95% CI (89–90%). In addition, mydriatic greater than two field strategy showed the highest level of sensitivity (92%, 95% CI 90–94) and specificity (94%, 95% CI 93–96%), a finding to be considered when setting up a screening strategy.
The optimum level of referable DR will depend on the accuracy of the screening strategy chosen and the resources available in the specific screening setting in order to strike a balance between screening PwDM at non-ophthalmic settings safely, but without overloading the eye clinics for further assessments. Annual DRS, followed by timely treatment of those confirmed to have STDR is the recommended screening pathway . The current method of DRS in most LMICs is an opportunistic screening using mydriatic bio-microscopic ophthalmoscopy by an ophthalmologist . This is not an efficient way of screening for DR considering the limitations in human resources and access barriers. In contrast, DRS using digital imaging requires specific training and skills, but these can be obtained by non-medical personnel, and as such the pool of potential workforce is much larger than for trained ophthalmologists.
In this meta-analysis, we aimed to show the effect of pupil status on DTA for any DR. For those images sets with gradable images, the pooled sensitivity of non-mydriatic strategies was the same as that of the mydriatic strategies. However, only six studies (6/21) used the same participants before and after pupil dilatation [25, 35, 40, 42, 44, 47]. The non-mydriatic method results were primarily dominated by one larger study (sample size n = 1549) conducted in a HIC  and another study used wide field (Optomap® 180–200° field view) imaging . Therefore, the outcome of this review should be applied to LMICs cautiously. A similar result was reported in a meta-analysis by Bragge et al. although heterogeneity among those studies was high due to pooling of different examination techniques in one estimation . In the current meta-analysis, heterogeneity was minimised by including studies which used digital retinal imaging only in the index test.
A DRS method which is suited to the health system is a key factor in the success of a programme. Non-mydriatic imaging can be used in settings where there are fewer ophthalmic personnel and avoiding pupil dilatation reduces screening time and causes less perceived inconvenience to PwDM. A concern, however, is variability in image quality, particularly in populations with a high prevalence of cataract and corneal opacities [14, 52]. The Scottish National Health Services DRSP now uses non-mydriatic imaging systems, with minimal need for pupil dilatation in screened patients . This is an evidence-based pragmatic approach with greater convenience for PwDM and lower cost to service providers [54, 55]. However, implementation of non-mydriatic test in DRS will depend on population characteristics such as the prevalence of cataract.
Selection of suitable personnel for DRS and grading depends on workforce capacity and availability. DRS by ophthalmologists is not an efficient way of screening for any setting . DM-related blindness is still on the rise everywhere in the world and is a public health concern in LMIC settings as well . These countries will have to rapidly adopt clinically safe and cost-effective strategies to address this issue, using the limited resources available and establish such a programme quickly . In this analysis, retinal image graders could achieve the recommended level of 80% sensitivity and specificity closer to 95% in both mydriatic and non-mydriatic strategies. Therefore, it is justifiable to train non-ophthalmic personnel in DR grading, just as it was done in the UK national programme.
DR screening’s success depends on the gradability of images, as such most of the studies included only gradable images. High population coverage with good quality gradable images is an important pragmatic consideration to achieve high DTA and high acceptability of a DRSP. Therefore, interpretation of the results shown in this study requires judgement of the context and objectives of a specific DRSP. PwDM with ungradable images are a special category of people whose fundus is not visible due to some other ocular pathology like dense lenticular opacities. These people therefore not only need the management that test negatives receive in terms of management of diabetic retinopathy but will also need additional management of ocular pathology which is obliterating the fundus image. Therefore, this meta-analysis highlights the concerns as to how to manage data on ungradable images, as studies differ in their approach of dealing with such a concern. Most authors (13 studies) had excluded ungradable images from their analysis while others included them as having screened positive (six studies). In addition, reporting of ungradable by study authors was heterogeneous, which imply requirement of standardised reporting of ungradable images in DRS.
The mean proportions of ungradable images in non-mydriatic and mydriatic imaging were 17.8% (95% CI 10.8–24.8%) and 6.1% (95% CI 3.7–8.4%) respectively. The decisions made by each study authors may have introduced reporting bias in their measures of DTA. Considering ungradable images as test positives may have led to inflated estimates of DTA in some studies [25, 26, 40, 42,43,44]. The mean proportions of ungradable images included by study authors as test positives in non-mydriatic and mydriatic imaging were 12.5% (95% CI 9.0–16.1%) and 2.5% (95% CI 1.0–3.9%) respectively. Therefore, we adjusted DTA to take account of ungradable images by excluding those to reduce heterogeneity. This was possible for four of the six studies in which ungradable images were included as screening positive [25, 26, 40, 43], but two did not report adequate data to allow for this [42, 44]. As an example, we made adjustment (calculated sensitivity 42/49 = 85.7%, specificity 227/262 = 86.6%) for the inflated DTA (reported sensitivity 98%, specificity 100%) in the study of Ahmed et al. using the 2 × 2 table data reported by study authors . In another two studies, it was not clear how ungradable images had been managed [28, 38]. The proportions of ungradable images and DTA after adjustments in each strategy are available in Additional file 9.
The definition of ungradable images was not uniform in the studies included in the current review We minimised the heterogeneity by excluding the ungradable images and by sub-group analysis.
The studies which used non-mydriatic imaging techniques were more recent, being conducted after rapid advancements in technology for such imaging technology leading to better quality images using non-mydriatic systems without pupil dilatation as well and a major confounder in the meta-analysis.
The results of the different strategies described in this review are to be considered fully if a comprehensive DRSP facilitating greater screening coverage with improved accessibility and good quality imaging is to be set up. However, due to lack of relevant good quality data, sub-analysis by countries’ income setting was not possible to perform due to absence of studies from LMICs.
We excluded three articles which were not in English due to practical barriers in translations and assessment of methodological quality.
The DTA of detection of maculopathy had not been considered. The maculopathy is also an important aspect in DRS, and it may have to be considered in a separate review.
Diagnostic test accuracy for the detection of any level of DR showed that DRS using two fields delivered at non-primary care settings is a feasible approach. Dilatation of the pupils did not improve the detection of any level of DR for those with gradable images, but such a wide range of ungradable were presented in these studies that this aspect must be taken into account when setting up DRSP. There was no adequate evidence in primary studies to comment on DTA of non-ophthalmological human resources on DRS, so this aspect requires further research. Good quality digital imaging has the potential for real-time interpretation of retinal images, which together with counselling for risk factors may improve the acceptability of DRS and uptake of referral for ophthalmic assessment if conducted in a culturally acceptable way.
Diagnostic test accuracies of the newer non-mydriatic imaging systems should be further explored in different environments and using a different skill-mix of graders, especially in LMICs.
Studies should focus on the accuracy of non-ophthalmic graders and non-ophthalmic settings to explore the potential of initiating DRSP especially in low-income settings. This will reduce the number of referrals to eye departments, many of which are already over-burdened with cataract and other eye conditions, particularly in LMIC where resources are limited.
The reporting definitions of technical failures or ungradability of the images should be standardised using a reporting guideline.
A systematic review and meta-analysis of DTA of different levels of DR and maculopathy can be recommended in future research.
Diagnostic test accuracy
Early treatment diabetic retinopathy study
Low- and middle-income country
Non-proliferative diabetic retinopathy
People with diabetes mellitus
Quality assessment for diagnostic accuracy studies
Sight-threatening diabetic retinopathy
Whiting DR, Guariguata L, Weil C, Shaw J. Global estimates of the prevalence of diabetes for 2011 and 2030. Diabetes Res Clin Pract. 2011;94(3):311–21. https://doi.org/10.1016/j.diabres.2011.10.029.
Diabetes - International Diabetes Federation. 2017. http://www.idf.org/idf-diabetes-atlas-seventh-edition. Accessed 10 Jan 2018.
Shaw JE, Sicree RA, Zimmet PZ. Global estimates of the prevalence of diabetes for 2010 and 2030. Diabetes Res Clin Pr. 2010;87(1):04–14. https://doi.org/10.1016/j.diabres.2009.10.007.
Guariguata L, Whiting DR, Hambleton I, Beagley J. Global estimates of diabetes prevalence for 2013 and projections for 2035. Diabetes Res Clin Pract. 2013;103(2):137–49. https://doi.org/10.1016/j.diabres.2013.11.002.
Klein R, Klein BEK. The epidemiology of diabetic retinopathy. Fifth Edit. Vols. 2–3, Retina: fourth edition. Elsevier Inc.; 2005. 1503-1521. doi: https://doi.org/10.1016/B978-1-4557-0737-9.00045-X.
Yau JWY, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, et al. Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care. 2012;35:556–64. https://doi.org/10.2337/dc11-1909.
Zhang X, Norris SL, Saadine J, Chowdhury FM, Horsley T, Kanjilal S, et al. Effectiveness of interventions to promote screening for diabetic retinopathy. Am J Prev Med. 2007;33(4):318–35. https://doi.org/10.1016/j.amepre.2007.05.002.
Klein BE. Overview of epidemiologic studies of diabetic retinopathy. Ophthalmic Epidemiol. 2007;14(4):179–83.
The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;329:977–86. https://doi.org/10.1056/NEJM199309303291401.
Klein R, Knudtson MD, Lee KE, Gangnon R, Klein BEK. The Wisconsin epidemiologic study of diabetic retinopathy xxii. The twenty-five-year progression of retinopathy in persons with type 1 diabetes. NIH public access. Diabetes. 2009;115(11):263–0279.
Kohner EM, Aldington SJ, Stratton IM, Manley SE, Holman RR, Mathews DR, Turner RC. United Kingdom Prospective Diabetes Study, 30: diabetic retinopathy at diagnosis of non-insulin-dependent diabetes mellitus and associated risk factors. Arch Ophthalmol. 1998;116(3):297–303.
The Diabetic Retinopathy Study Research Group. Photocoagulation treatment of proliferative diabetic retinopathy. Ophthalmology. 1976;88(7):583–600. https://doi.org/10.1016/S0161-6420(81)34978-1.
Scanlon PH. The English National Screening Programme for diabetic retinopathy 2003–2016. Acta Diabetol. 2017;54(6):515–25. https://doi.org/10.1007/s00592-017-0974-1.
Banaee T, Ansari-Astaneh MR, Pourreza H, Faal Hosseini F, Vatanparast M, Shoeibi N, et al. Utility of 1% tropicamide in improving the quality of images for tele-screening of diabetic retinopathy in patients with dark irides. Ophthalmic Epidemiol. 2017;24(4):217–21. https://doi.org/10.1080/09286586.2016.1274039.
Hutchinson A, McIntosh A, Peters J, Keeffe JE, Khunti K, Baker R. Effectiveness of screening and monitoring tests for diabetic retinopathy - a systematic review. Diabet Med. 2000;17(17):495–506 PMID: 10972578.
Shi L, Wu H, Dong J, Jiang K, Lu X, Shi J. Telemedicine for detecting diabetic retinopathy: a systematic review and meta-analysis. Br J Ophthalmol. 2015;99(6):823–31. https://doi.org/10.1136/bjophthalmol-2014-305631.
Bragge P, Gruen RL, Chau M, Forbes A, Taylor HR. Screening for presence or absence of diabetic retinopathy. Arch Ophthalmol. 2011;129(4):435–44. https://doi.org/10.1001/archophthalmol.2010.319.
Lin S, Ramulu P, Lamoureux EL, Sabanayagam C. Addressing risk factors, screening, and preventative treatment for diabetic retinopathy in developing countries: a review. Clin Exp Ophthalmol. 2016;(October 2015):300–20. https://doi.org/10.1111/ceo.12745 PMID: 26991970.
Hipwell AE, Sturt J, Lindenmeyer A, Stratton I, Gadsby R, Hare PO. Attitudes, access and anguish: a qualitative interview study of staff and patients ’ experiences of diabetic retinopathy screening. BMJ Open. 2014;4:e005498. https://doi.org/10.1136/bmjopen-2014-005498.
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PCVJSI. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med. 2007;4(10):e296. https://doi.org/10.1371/journal.pmed.
Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y. Cochrane handbook for systematic reviews of diagnostic test accuracy chapter 10 analysing and presenting results. Cochrane DTA Handb. 2010:1–61 http://methods.cochrane.org/sites/methods.cochrane.org.sdt/files/public/uploads/Chapter%2010%20-%20Version%201.0.pdf. Accessed 30 Sept 2015.
Whiting PF, Rutjes AWS, Westwood ME, Mallet S, Deeks JJ, Reitsma JB, et al. Research and reporting methods accuracy studies. Ann Intern Med. 2011;155(4):529–36. https://doi.org/10.7326/0003-4819-155-8-201110180-00009.
Herbert HM, Jordan K, Flanagan DW. Is screening with digital imaging using one retinal view adequate? Eye (Lond). 2003;17(4):497–500. https://doi.org/10.1038/sj.eye.6700409.
Olson JA, Strachan FM, Hipwell JH, Goatman KA, McHardy KC, Forrester JV, et al. A comparative evaluation of digital imaging, retinal photography and optometrist examination in screening for diabetic retinopathy. Diabet Med. 2003;20(7):528–34 PMID: 12823232.
Hansen AB, Sander B, Larsen M, Kleener J, Borch-Johnsen K, Klein R, et al. Screening for diabetic retinopathy using a digital non-mydriatic camera compared with standard 35-mm stereo colour transparencies. Acta Ophthalmol Scand. 2004;82(6):656–65. https://doi.org/10.1111/j.1600-0420.2004.00347.x PMID: 15606460.
Neubauer AS, Kernt M, Haritoglou C, Priglinger SG, Kampik A, Ulbig MW. Nonmydriatic screening for diabetic retinopathy by ultra-widefield scanning laser ophthalmoscopy (Optomap). Graefes Arch Clin Exp Ophthalmol. 2008;246(2):229–35. https://doi.org/10.1007/s00417-007-0631-4 PMID: 17622548.
Henricsson M, Karlsson C, Ekholm L, Kaikkonen P, Sellman A, Steffert E, et al. Colour slides or digital photography in diabetes screening--a comparison. Acta Ophthalmol Scand. 2000;78(2):164–8. https://doi.org/10.1034/j.1600-0420.2000.078002164.x/ PMID: 10794249.
Sundling V, Gulbrandsen P, Straand J. Sensitivity and specificity of Norwegian optometrists’ evaluation of diabetic retinopathy in single-field retinal images - a cross-sectional experimental study. BMC Health Serv Res. 2013;13:17. https://doi.org/10.1186/1472-6963-13-17.
Ahmed J, Ward TP, Bursell SE, Aiello LM, Cavallerano JD, Vigersky RA. The sensitivity and specificity of nonmydriatic digital stereoscopic retinal imaging in detecting diabetic retinopathy. Diabetes Care. 2006;29(10):2205–9. https://doi.org/10.2337/dc06-0295.
Shiffman R, Jacobsen G, Nussbaun J, Desai U, Carey D, Glasser D, et al. Comparison of a digital retinal imaging system and seven-field stereo colour fundus photography to detect diabetic retinopathy in the primary care environment. Ophthalmic Surg Lasers Imaging. 2005;36:46–57 PMID: 15688971.
Boucher MC, Gresset JA, Angioi K, Olivier S. Effectiveness and safety of screening for diabetic retinopathy with two nonmydriatic digital images compared with the seven standard stereoscopic photographic fields. Can J Ophthalmol. 2003;38(7):557–68 PMID: 14740797.
Maberley D, Cruess AF, Barile G, Slakter J. Digital photographic screening for diabetic retinopathy in the James Bay Cree. Ophthalmic Epidemiol. 2002;9(3):169–78 PMID: 12045884.
Perrier M, Boucher MC, Angioi K, Gresset JA, Olivier S. Comparison of two, three and four 45 degrees image fields obtained with the Topcon CRW6 nonmydriatic camera for screening for diabetic retinopathy. Can J Ophthalmol. 2003;38(7):569–74 PMID: 14740798.
Bhargava M, Cheung CYL, Sabanayagam C, Kawasaki R, Harper CA, Lamoureux EL, et al. Accuracy of diabetic retinopathy screening by trained non-physician graders using non-mydriatic fundus camera. Singap Med J. 2012;53(11):715–9 PMID: 23192497.
Murgatroyd H. Effect of mydriasis and different field strategies on digital image screening of diabetic eye disease. Br J Ophthalmol. 2004;88(7):920–4. https://doi.org/10.1136/bjo.2003.026385 PMID: 15205238 PMCID: PMC1772219.
Mizrachi Y, Knyazer B, Guigui S. Evaluation of diabetic retinopathy screening using a non-mydriatic retinal digital camera in primary care settings in south Israel. Int Ophthalmol. 2014:831–7. https://doi.org/10.1007/s10792-013-9887-3 PMID: 24292883.
Ku JY, Landers J, Henderson T, Craig JE. The reliability of single-field fundus photography in screening for diabetic retinopathy: the central Australian ocular health study. Med J Aust. 2013;198(2):93–5. https://doi.org/10.5694/mja12.10607.
Phiri R, Keeffe JE, Harper CA, Taylor HR. Comparative study of the polaroid and digital non-mydriatic cameras in the detection of referrable diabetic retinopathy in Australia. 2006:867–72. https://doi.org/10.1111/j.1464-5491.2006.01824.x.
Scanlon PH, Malhotra R, Greenwood RH, Aldington SJ, Foy C, Flatman M, et al. Comparison of two reference standards in validating two field mydriatic digital photography as a method of screening for diabetic retinopathy. Br J Ophthalmol. 2003;87(10):1258–63 PMID: 14507762 PMCID: PMC1920793.
Scanlon PH, Malhotra R, Thomas G, Foyt C, Kirkpatrick JN, Lewis-Barned N, et al. The effectiveness of screening for diabetic retinopathy by digital imaging photography and technician ophthalmoscopy. Diabet Med. 2003;20(6):467–74 PMID: 12786681.
Tu K, Palmer P, Sen S, Matthew P, Khaleel A. Comparison of optometry vs digital photography screening for diabetic retinopathy in a single district. Eye. 2004;18(July 2002):3–8. https://doi.org/10.1038/sj.eye.6700497.
Aptel F, Denis P, Rouberol F, Thivolet C. Screening of diabetic retinopathy: effect of field number and mydriasis on sensitivity and specificity of digital fundus photography. Diabetes Metab. 2008;34(3):290–3. https://doi.org/10.1016/j.diabet.2007.12.007.
Massin P, Erginay A, Ben Mehidi A, Vicaut E, Quentel G, Victor Z, et al. Evaluation of a new non-mydriatic digital camera for detection of diabetic retinopathy. Diabet Med. 2003;20(8):635–41.
Baeza M, Orozco-Beltrán D, Gil-Guillen VF, Pedrera V, Ribera MC, Pertusa S, et al. Screening for sight threatening diabetic retinopathy using non-mydriatic retinal camera in a primary care setting: to dilate or not to dilate? Int J Clin Pract. 2009;63(3):433–8. https://doi.org/10.1111/j.1742-1241.2008.01921.x.
Lopez-Bastida J, Cabrera-Lopez F, Serrano-Aguilar P. Sensitivity and specificity of digital retinal imaging for screening diabetic retinopathy. Diabet Med. 2007;24(4):403–7. https://doi.org/10.1111/j.1464-5491.2007.02074.x.
Suansilpong A, Rawdaree P. Accuracy of single-field nonmydriatic digital fundus image in screening for diabetic retinopathy. J Med Assoc Thail. 2008;91(9):1397–403 PMID: 18843870.
Ding J, Zou Y, Liu N, Jiang L, Ren X, Jia W, et al. Strategies of digital fundus photography for screening diabetic retinopathy in a diabetic population in urban China strategies of digital fundus photography for screening diabetic retinopathy in a diabetic population in urban China. Ophthalmic Epidemiol. 2016;6586. https://doi.org/10.3109/09286586.2012.716895.
Kuo HK, Hsieh HH, Liu RT. Screening for diabetic retinopathy by one-field, non-mydriatic, 45° digital photography is inadequate. Ophthalmologica. 2005;219(5):292–6. https://doi.org/10.1159/000086114.
Massin P, Aubert JP, Eschwege E, Erginay A, Bourovitch JC, BenMehidi A, et al. Evaluation of a screening program for diabetic retinopathy in a primary care setting Dodia (Dépistage ophtalmologique du diabète) study. Diabetes Metab. 2005;31(2):153–62 PMID: 15959421.
Mead A, Burnett S, Davey C. Diabetic retinal screening in the UK. J R Soc Med. 2001:127–9. https://doi.org/10.1177/014107680109400307.
Taylor-Phillips S, Mistry H, Leslie R, Todkill D, Tsertsvadze A, Connock M, et al. Extending the diabetic retinopathy screening interval beyond 1 year: systematic review. Br J Ophthalmol. 2016;100(1):105–14. https://doi.org/10.1136/bjophthalmol-2014-305938.
Scanlon P, Foy C, Malhotra R, Aldington S. The influence of age, duration of diabetes, cataract, and pupil size on. Diabetes Care. 2005;28(10) PMID: 16186278.
The Diabetic Retinopathy Screening Implementation Group - Diabetic retinopathy screening services in Scotland: recommendations for implementation. 2003. Available from: http://www.ndrs-wp.scot.nhs.uk/wp-content/uploads/2013/04/Recommendations-for-Implementing-DRS.pdf Accessed on 15 Sept 2017.
Guigui S, Lifshitz T, Levy J. Screening for diabetic retinopathy: review of current methods. Hosp Pract. 2012;40(2):64–72. https://doi.org/10.3810/hp.2012.04.971 PMID: 22615080.
Swanson M. Retinopathy screening in individuals with type 2 diabetes: who, how, how often, and at what cost--an epidemiologic review. Optometry. 2005;76(11):636–46. https://doi.org/10.1016/j.optm.2005.08.019 PMID: 16298316.
Khandekar R. Screening and public health strategies for diabetic retinopathy in the Eastern Mediterranean Region. Middle East Afr J Ophthalmol. 2012;19(2):178. https://doi.org/10.4103/0974-9233.95245 PMCID: PMC3353664. PMID: 22623855.
This review was a formative component of a research degree student project funded by Queen Elizabeth Diamond Jubilee Trust, UK, coordinated through Commonwealth Eye Health Consortium, UK.
Availability of data and materials
All data generated or analysed during this review are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
Ethics approval was obtained from Research Ethics Committee of London School of Hygiene and Tropical Medicine, UK.
Consent for publication
The authors provided an approval for publication in BMC Systematic Review.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
PRISMA check list. (DOCX 21 kb)
Details of the excluded studies. (DOCX 49 kb)
Participant characteristics of the included articles. (DOCX 28 kb)
DTA of different strategies and ungradable image proportions as reported by study authors. (DOCX 29 kb)
DTA parameters by pupil status and field strategy using HSROC curves. (DOCX 20 kb)
Forest plots of DTA variation by type of reference standard and by the level of service delivery (by clinic settings). (DOCX 536 kb)
Forest plots of DTA by different index test human resources. (DOCX 304 kb)
Forest plots of sub-analyses—DTA using same participant undergoing imaging before and after pupil dilatation. (DOCX 162 kb)
DTA following adjustments in relevant to exclusion of ungradable proportions in the current review. (DOCX 27 kb)
About this article
Cite this article
Piyasena, M.M.P.N., Murthy, G.V.S., Yip, J.L.Y. et al. Systematic review and meta-analysis of diagnostic accuracy of detection of any level of diabetic retinopathy using digital retinal imaging. Syst Rev 7, 182 (2018). https://doi.org/10.1186/s13643-018-0846-y
- Diabetes mellitus
- Diabetic retinopathy
- Diagnostic test accuracy
- Digital imaging
- Low income