- Research
- Open Access
- Published:
Prognostic models of diabetic microvascular complications: a systematic review and meta-analysis
Systematic Reviews volume 10, Article number: 288 (2021)
Abstract
Background
Many prognostic models of diabetic microvascular complications have been developed, but their performances still varies. Therefore, we conducted a systematic review and meta-analysis to summarise the performances of the existing models.
Methods
Prognostic models of diabetic microvascular complications were retrieved from PubMed and Scopus up to 31 December 2020. Studies were selected, if they developed or internally/externally validated models of any microvascular complication in type 2 diabetes (T2D).
Results
In total, 71 studies were eligible, of which 32, 30 and 18 studies initially developed prognostic model for diabetic retinopathy (DR), chronic kidney disease (CKD) and end stage renal disease (ESRD) with the number of derived equations of 84, 96 and 51, respectively. Most models were derived-phases, some were internal and external validations. Common predictors were age, sex, HbA1c, diabetic duration, SBP and BMI. Traditional statistical models (i.e. Cox and logit regression) were mostly applied, otherwise machine learning. In cohorts, the discriminative performance in derived-logit was pooled with C statistics of 0.82 (0.73‑0.92) for DR and 0.78 (0.74‑0.83) for CKD. Pooled Cox regression yielded 0.75 (0.74‑0.77), 0.78 (0.74‑0.82) and 0.87 (0.84‑0.89) for DR, CKD and ESRD, respectively. External validation performances were sufficiently pooled with 0.81 (0.78‑0.83), 0.75 (0.67‑0.84) and 0.87 (0.85‑0.88) for DR, CKD and ESRD, respectively.
Conclusions
Several prognostic models were developed, but less were externally validated. A few studies derived the models by using appropriate methods and were satisfactory reported. More external validations and impact analyses are required before applying these models in clinical practice.
Systematic review registration
PROSPERO CRD42018105287
Background
Type 2 diabetes (T2D) has increased rapidly over the past 30 years becoming worldwide public health problem with prevalence in adults of 463 million (9.3%) in 2019. It is estimated to be 700 million (10.9%) by 2045 [1], in which currently about 79% of people have diabetes living in low- and middle-income countries [1, 2]. Furthermore, diabetic progression due to its complications-increased disability, impaired quality of life and leading cause of premature death, which accounted for 11.3% of the global mortality [1, 3].
Two life threatening microvascular complications in T2D are diabetic retinopathy (DR) and diabetic nephropathy (DN). DN, known as chronic kidney disease (CKD), characterised by proteinuria and rapidly declined glomerular filtration rate (GFR) [2, 4], accounted for approximately 20‑40% of diabetic populations [5]. DR is the major cause of blindness [6] through fractional retinal detachment, preretinal or vitreous haemorrhage and central vision impairment, with the prevalence of 25% globally [7].
Diabetic microvascular complications commonly occurs in working ages [8], thus declining productivity, increasing tremendous social cost and high burden in healthcare [9, 10]. Therefore, early identification of high-risk patients to prevent occurrence of microvascular complications is very important. Many prognostic models have been developed (e.g. DR [11,12,13,14,15,16,17,18], CKD [19,20,21,22,23,24,25] and ESRD models [12, 26,27,28,29]) using various statistical methods. A lot of prognostic models were externally validated [12, 17, 19, 20, 22, 26, 29, 30], whilst other models were not [13, 15, 16, 21, 27, 28]. Nonetheless, the best prognostic model for each complication was still inconclusive. Hence, we conducted a systematic review to summarise all prognostic models for diabetic microvascular complications (including DR, CKD and ESRD) that are available and their performances in prediction of complications.
Methods
Protocol registration
This study was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [31] and in accordance with CHARMS checklist [32]. The review protocol was registered at PROSPERO (CRD42018105287).
Search strategy
Studies were identified from PubMed and Scopus up to 31 December 2020. Search terms were constructed based on patients, interventions and outcomes, see details in Additional files 1 and 2.
Study selection
Studies, published in any language, were eligible if they studied in adult T2D, developed or validated any multivariable prognostic models of microvascular complications in T2D with applying any traditional statistical modelling (e.g. logit or Cox regression etcetera) or machine learning (ML), and reported model performance. We also included the studies from reference list of relevant publications.
Data extraction
Data extractions were performed by one reviewer (SAS) and checked by OP. Extracted data were characteristics of study and patients (i.e. country, study design, settings, data source, sample size and number of events, ethnicity, age, percent male and diabetic duration), study phase (i.e. derivation or validation), statistical methods, predictors, missing data and outcomes (i.e. DR, CKD and ESRD). In addition, two related properties of model performances (calibration and discrimination) were also extracted.
Risk of bias assessment
Risk of bias assessment was assessed by using Prediction Model Risk of Bias Assessment Tool (PROBAST) [32]. Each item was rated as low, high or unclear. The overall validity was low and high risk if all domains were low risk, and at least one domain was high risk, respectively. Discrepancies were solved by consensus between the team.
Statistical analysis
Characteristics of each prognostic model and predictive performances (including calibration and discrimination) were described. Discrimination was assessed according to original included studies, in which C-statistic was mostly used. If studies reported C-statistic without variance, it was estimated using equations in the previous guidelines [32,33,34]. Calibration was assessed [35] using calibration plot, goodness-of-fit testing (i.e. Hosmer and Lemeshow χ2 test), calibration slopes or the observed/expected (O/E) ratio.
A meta-analysis was applied for pooling C statistics across studies stratified by study’s design/phase, statistical model and T2D complications. A random-effect model by DerSimonian-Laird [36, 37] was used if heterogeneity was present (p value < 0.10 or I2 > 25%); otherwise, a fixed-effect model was used. A heterogeneity was assessed by Cochrane Q test and I2 statistic. Publication bias in external validation was assessed using funnel-plot [38] and Egger’s test [39]. All statistical analyses were performed using STATA 16 [40]. A p value less than 0.05 was considered as statistically significant, except for heterogeneity which used 0.10.
Results
A total of 32/1009 and 44/3321 studies were eligible for DR and DN, respectively, see Table S1 (Figs. S1‑S2). Amongst them, 205 prognostic equations were derivative, some of them performed internal and external validations. Most studies reported C statistics, but only a few-portions reported calibrations (Table S2).
Risk of bias assessment
Risk of bias assessment of all included studies was presented in Table S3. Amongst 71 studies, about 86 to 95% of studies were determined as low risk of bias for study participants, selection of predictors and outcome measurement. About 23% and 40% of studies were rated as high risk for sample size, participant flow and statistical analysis, respectively. As a result, 35% of studies were overall low risk of bias (Fig. S3).
Diabetic retinopathy
Thirty-two [11,12,13,14,15,16,17,18, 24, 30, 41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62] studies were identified for predictions of DR including 1,120,278 diabetic patients with 128,129 (11.4%) events. Of which, 26 [11, 12, 14, 18, 24, 30, 41 50, 53, 62] and 6 [13, 15,16,17, 51, 52] studies applied traditional statistical modelling and ML, respectively. Twenty-eight [11,12,13,14,15,16,17,18, 24, 41, 42, 45, 47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62] studies derived 84 original prognostic equations with varied sample sizes of 18 to 254,896. Mean age was 44.6 to 66.6 years, percent male was 27.0 to 61.9, and diabetic duration varied from 1.4 to 15.8 years. Most studies were conducted in Europe and America regions with only 11 [17, 42, 43, 45, 47, 49, 52,53,54,55, 59] (34%) studies in Asia. Twenty-four [11,12,13,14,15,16, 18, 24, 30, 41,42,43, 45,46,47,48, 51, 52, 54, 56, 58,59,60, 62] and 8 [17, 44, 49, 50, 53, 55, 57, 61] studies were hospital and community-based settings respectively with confirmation of T2D diagnosis from medical records, laboratory tests or use of diabetic drugs. DR was mostly diagnosed by using fundus examination. Follow-up time ranged varied from 1.0 [56] to 20 [11] years with a median of 5 years. Only 5 [18, 24, 30, 50, 58] studies reported percent loss to follow-up which ranged from 2.4 to 31.3%. Eighteen [11,12,13, 15, 30, 41,42,43,44,45, 48, 50, 51, 54, 55, 57, 59, 61] studies used various methods for dealing with missing data, in which about a half of them used multiple imputations (Table S1). Four [45, 47, 54, 57] and 3 [13, 18, 49] studies provided simplified scoring system and presented nomograms, or else only used regression coefficients/odds ratio to calculate the score.
As for phase of prediction, 4 [12, 17, 41, 42], 2 [11, 45], 4 [30, 43, 44, 46], 15 [13, 15, 16, 18, 48-54, 56‑59] and 6 [14, 24, 55, 60, 62, 63] studies were respectively determined as derived-internal-external (D/I/E), derived-external (D/E), external (E), derived-internal (D/I) and only derived (D) phases (Table S2). Amongst 8 [12, 17, 30, 41,42,43,44, 46] external-validation studies, 5 [12, 17, 41, 42, 44] validated their own derived models in the same ethnicity (i.e. Asian [17, 42], mixed ethnicity [12, 41, 44]), except 3 [30, 43, 46] that validated other author’s models that were originally in Caucasians [11] and Asians [45, 47]. Most studies were cohorts/RCT’s, and their DR incidence varied from 1.5% [48] to 42.9% [56] whereas it was 14.2% [43] to 57.2% [54] in cross-sectional studies.
Seventeen [13, 14, 17, 42, 43, 45, 47, 49, 51,52,53,54,55,56, 58, 61, 64] and 9 [11, 12, 18, 24, 41, 44, 46, 59, 60] studies used logit and Cox whereas 6 [13, 15,16,17, 51, 52] applied MLs. Various predictors were considered (Fig. S4), in which the most commonly used were diabetic duration, age, HbA1c, SBP and BMI, which these were mainly included in the models as continuous predictors. A total number of included predictors in the conventional statistical models and MLs were not much different with a median of 8 (range 2‑37). Interestingly, few studies used image/signal analytic [56, 58] and genetic variables [42, 55, 62], which were incorporated with conventional clinical data. Two [56, 58] studies predicted specific DR site using multifocal electroretinogram incorporated with traditional clinical factors [56, 58].
Three [42, 55, 62] studies derived genetic risk score (GRS) based on different genetic polymorphisms (range 2‑76). Traditional prognostic factors (i.e. age, sex, diabetic duration, HbA1c and hypertension/SBP) were also retained in the model with GRS.
C statistic varied from 0.50 [13] to 0.95 [58], 0.52 [17] to 0.92 [58] and 0.59 [12] to 0.83 [30] for derived, internal and external validations. Those prognostic equations had been externally validated with moderate to good performance (Table S4). Discrimination performance of logit equations varied from moderate to high with the C statistics of 0.70 [45, 47, 49, 62] to 0.95 [58] and 0.63 [47] to 0.92 [58] in derived and internal validation; likewise, for support vector ML [17] in these corresponding phases of 0.83 and 0.81.
Pooled C statistics of the derived-logit models across cohorts [13, 51, 56,57,58, 61], cross-sectional [14, 17, 45, 47, 49, 52, 53] and case-control genetic [42, 55, 62] studies were 0.82 (0.73‑0.92; I2 = 99.47%), 0.77 (0.72‑0.82; I2 = 93.21%) and 0.74 (0.71‑0.77; I2 = 36.73%), respectively (Fig. 1). Fixed effect model was observed on pooled derived-Cox [11, 18, 24] models in cohort studies which yielded 0.75 (0.74‑0.77; I2 = 0.0%).
Pooled C statistics of logit equations across cohorts [13, 51, 56,57,58] and cross-sectional [17, 47, 49, 52,53,54] studies for internal validation were 0.83 (0.76‑0.90; I2 = 95.11%) and 0.74 (0.68‑0.81; I2 = 82.46%), respectively. Of those, the external validation for three [17, 45, 47] equations yielded 0.81 (0.78‑0.83; I2 = 8.05%) by cross-sectional studies of logit regression (Table S5). Funnel plot and Egger’s test (p > 0.293) suggested no publication bias by the absence of small study effects, no correlation between sample size and the magnitude of C statistics in external validation studies (Fig. S5).
Model’s calibration in derived phase [11, 12, 14, 18, 49, 54, 59, 61], internal [12, 18, 41, 49, 54, 59] and external [12, 30, 41, 44, 46] validations mostly demonstrated perfect O/E. Nine studies [14,15,16,17, 24, 52, 56, 58, 61] might have overfitted model as the ratio of an event per variable (EPV) numbers ranged from 1 [58] to 9 [24, 52], whereas the other 2 [12, 45] models might have underfitted with a ratio as high as 297 [12] to 403 [45].
Diabetic nephropathy
CKD
Thirty [12, 13, 19,20,21,22,23,24,25, 44, 59, 65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83] studies purely derived 96 equations including 244,934 diabetic patients with 44,023 (17.9%) events of CKD. CKD incidence ranged from 12.1 to 37.3% for 5 RCT’s [12, 59, 70, 74, 77], and 0.7% to 47.6% for 22 [12, 13, 20, 22,23,24,25, 65, 66, 68, 69, 71,72,73, 75, 76, 78,79,80,81,82,83] cohorts. Eleven studies [19, 22, 25, 59, 65,66,67, 69, 76, 78, 82] (36.7%) were conducted in Asians. Sixteen [13, 21, 23, 66,67,68,69, 71, 72, 75, 76, 78, 79, 81,82,83] and 16 [12, 19, 20, 22, 24, 25, 44, 59, 65, 69,70,71, 73, 74, 77, 80] studies diagnosed CKD based on eGFR and albuminuria, respectively. Median (range) follow-up time was 5.4 (1‑10) years and percent lost to follow up was 0.8% to 35.7%. Twelve [12, 13, 20, 22, 25, 44, 59, 65, 66, 69, 73, 80] (38.7%) studies reported methods for dealing with missing data, 5 [12, 22, 44, 59, 80] had used multiple imputations and only a few reported percent missing data [12, 13, 20, 44].
Five [12, 19, 23, 68, 71], 2 [20, 25], 2 [44, 69], 12 [13, 22, 59, 65,66,67, 70, 72,73,74, 78, 79] and 9 [21, 24, 75,76,77, 80,81,82,83] studies were D/I/E, D/E, E, D/I and only D-phases, respectively. Of 9 E-phases, eight [12, 19, 20, 23, 25, 44, 68, 71] studies validated their own models in different datasets, and one [69] validated others author’s model (i.e. QKIDNEY risk score), which developed in general populations. Half of the studies were validated in Asians [19, 20, 25, 43], and 9 [12, 20, 23, 25, 44, 68, 69, 71] used data from cohorts/RCTs. Their mean age ranged from 44.0 to 67.3 years, whereas the percent of male varied from 32.5 to 76.0 with a median follow-up time of 4.9 years.
Out of 28 studies, 96 derived models consisted of 79 traditional statistical models (i.e. logit (n = 16) [13, 19-21, 23, 25, 65‑68, 70, 72, 73, 78, 79, 82] and Cox (n = 11) [12, 22, 24, 59, 74,75,76,77, 80, 81, 83]), whereas 17 models (n = 4) [13, 70, 71, 73] performed various MLs algorithms (Table S2). Three [13, 65, 66] studies provided nomograms, whereas 2 [25, 74] studies simplified risk score.
Ninety-two derived models reported C statistics, with 55 (59.7%) internal and 19 (20.6%) external validations. Their discriminative performance varied from 0.50 [13] to 0.93 [21, 66], 0.50 [13] to 0.91 [73] and 0.57 [19] to 0.85 [44] in derived, internal and external phases, respectively (Table S2), which were explicitly described (Table S4). Common predictors were SBP, HbA1c, sex, diabetic duration and eGFR. Two [19, 67] studies combined genetic factors with clinical factors which yielded better discrimination of 0.78 (0.75‑0.81) relative to considered conventional models 0.75 (0.72‑0.78), see Table S6.
Out of twenty-eight [12, 13, 19,20,21,22,23,24,25, 59, 65,66,67,68, 70,71,72,73,74,75,76,77,78,79,80,81,82,83] derived studies, 14 [20, 22, 23, 59, 65,66,67,68, 72,73,74, 78,79,80] studies reported acceptable calibration model with O/E ratio ranged from 0.77 [79] to 1.11 [73]. Only seven [12, 23, 67, 73, 74, 78, 79] and six [12, 20, 22, 44, 68, 69] out of 21 validated-studies had O/E ratio of 0.93 [78] to 1.14 [12] and 0.97 [44] to 1.31 [12] in I and E phases (Table S2).
For cohorts, the pooled C statistics for logit (n = 11) [13, 20, 23, 25, 65, 66, 68, 72, 78, 79, 82] and Cox (n = 7) [22, 24, 75, 76, 80, 81, 83] in D phase were 0.78 (0.74‑0.83; I2 = 96.91%) and 0.78 (0.74‑0.82; I2 = 91.78%), respectively (Fig. 2). Cox regression in derived RCTs (n = 3) [59, 74, 77] yielded pooled C statistics of 0.73 (0.62‑0.84; I2 = 95.53%).
Derived logit (n = 8) [13, 23, 65, 66, 68, 72, 78, 79] from cohorts were internally validated yielding the pooled C statistics of 0.79 (0.74‑0.83; I2 = 95.53%) which performed closely to the D phases, but poorer in externally validated [20, 23, 25, 68] with pooled C statistic of 0.75 (0.67‑0.84; I2 = 94.68%). Funnel plot and Egger’s test (p > 0.710) showed no publication bias by the absence of small study effects (Fig. S6). There is no correlation between studies for smaller cohorts with higher C statistics.
ESRD
Eighteen studies [12, 26,27,28,29, 44, 72, 74, 84,85,86,87,88,89,90,91,92,93] originally derived 46 models in 366,210 diabetic patients with the ESRD incidence of 57,294 (15.65%). Of them, 13 [26, 28, 29, 44, 72, 84,85,86,87, 89,90,91,92] and 5 [12, 27, 74, 88, 93] were cohorts and RCTs, respectively. A half of them were conducted in Asia [28, 29, 84, 85, 89, 90, 92] and the USA [12, 27, 44, 72, 86, 87, 91]. Thirteen [12, 26,27,28,29, 72, 74, 84, 88,89,90,91, 93] and 5 [44, 85,86,87, 92] studies were hospital-based and community-based settings respectively, where ESRD was mostly confirmed by dialysis [12, 26,27,28, 44, 72, 88,89,90,91,92]. Mean follow-up times ranged from 1.5 to 14 years. Only 9 studies (50%) reported methods for dealing with missing data, in which 5 [12, 27, 29, 44, 84] used multiple imputations. Two studies [74, 84] developed simplified risk score based on the Framingham Heart study [94].
Briefly, 1 [12], 2 [26, 29], 1 [44], 5 [28, 72, 74, 84, 85] and 9 [27, 86,87,88,89,90,91,92,93] studies showed D/I/E, D/E, E, D/I, and only D phases, respectively (Table S1). Three [12, 26, 44] studies have externally validated their own models within the same studies, whereas 1 [29] study validated other models’ studies [26, 74].
C statistics varied from 0.76 [89] to 0.97 [27, 85] in derivative phases of 17 studies (n = 51), 0.60 [12] to 0.96 [85] in internal validations (n = 11) and 0.54 [12] to 0.92 [26] in external validations (n = 13), see Table S2. Prognostic model for ESRD was mainly derived by Cox equation in 16 (95%) studies [12, 26‑29, 74, 84-93].
The pooled C statistics in cohorts using Cox were 0.87 (0.84‑0.89; I2 = 92.15%), 0.91 (0.86‑0.96; I2 = 94.86%) for derived and internal validations suggesting discrimination in validations were not much different compared with derived phases. The pooled C statistics in derived RCTs were 0.88 (0.78‑0.98; I2 = 96.82%), see Fig. 3. Moreover, pooled C statistics in external validation demonstrated good performance of 0.86 (0.85‑0.88) in 3 [26, 29, 44] cohorts (Table S5). Funnel plot and Egger’s test (p > 0.513) showed no publication bias by the absence of small study effects in external validation studies for predicting ESRD (Fig. S7).
Common predictors for ESRD were age, sex, HbA1c, eGFR and BMI (Table S6). Predictive models of 6 [27, 28, 86, 87, 91] studies might have over-fitted as events/variable for applying Cox resulted in 6 [28] to 9 [92], whereas their ratios in 4 [12, 29, 89, 90] other studies observed a rule of thumb of 10‑20 (Fig. 3, Table S2).
Discussion
This review summarised prognostic models that were developed and validated for predicting microvascular complications (i.e. DR, CKD and ESRD) in T2D patients. Model performances were described prognostic models separately by derived, internal and external validation.
Seven predictors were commonly used in predictive models of DR, DN and ESRD including age, sex, BMI, diabetic duration, HbA1c, SBP and eGFR. The DR models showed well discriminated with pooled C statistics of 0.82, 0.83 and 0.81 in D, I and E validations, respectively. Model performance was only moderate‑good in CKD for all phases (i.e. the corresponding pooled C statistics about 0.78, 0.79 and 0.75) but quite excellence for ESRD models (i.e. 0.87, 0.91 and 0.86, respectively).
Only a few prognostic models were externally validated with moderate to good discrimination performance, which are applicable in clinical practice. For instance, a few DR-models [11, 12, 41] had good discrimination and calibration in external validations. Three [12, 20, 68] DN models had good discrimination with fair calibration. Other three [12, 26, 29] ESRD models with very large size cohorts were generalisable with good discriminations and were even developed in different ethnicities. Calibration performance was less reported relative to discrimination, although both parameters should be reported for prognostic model development [95,96,97]. Particularly for observed to expected (O/E) ratio was reported in very few studies, which prevented meta-analysis of calibration.
Currently many prediction models are available by online calculators, or differently presented simplified risk scores or nomograms. Some online risk-calculators have been developed to simplify knowledge translation in clinical practice (i.e. DR [11, 12, 41, 59], CKD [12, 20] and ESRD [26]). However, very few of them have been applied due to the absence of some predictors and users’ interpretations in routine health practice.
We found various clinical settings and developed equations, but only few of them were externally validated with insufficiently reported with a wide range of CKD definitions. Amongst them, there might be potentially over-optimistic as EPV was less than ten by the rule of thumb in a regression model. None of the studies performed impact assessments by applying prognostic models into clinical practice.
Numerous predictors were simultaneously included in the prognostic models with a median of 8 (IQR, 5‑10) predictors. In brief, demographics, biomarkers and clinical features were commonly considered for derived-models of micro-vascular complications. Medical treatment (including anti-hypertensive and diabetic drug control) and some comorbidities were included into some derived equations. Likewise, DR itself might be a predictor of DN [20, 22, 66, 74, 75]. Interestingly, nonconventional predictors (i.e. genetic and image processing [56, 58]) could also predict DR [42, 55, 62] and DN [19, 67].
Missing data in clinical settings particularly for routine datasets are unavoidable. Frequently, the investigators only performed complete-case analysis. Handling missing data is vitally important to prevent biassed results and lost power in generalisations [98]. Additionally, categorisation of continuous predictors or dichotomisation may result in missing information, significant misleading [99], incorrect variable selection and may decrease prediction accuracy [100, 101].
Cohort or RCT should be the most appropriated design for developing prognostic model, whereas a cross-sectional study could be used for external validation. Exceptionally, nested case-control and case-cohort studies were still applicable [96]. The rule of thumb suggested that a number of 10‑20 events should be available for one predictor in a multivariable logit/Cox regression [96, 102, 103]. For instance, seven [14, 17, 24, 52, 56, 58, 61] studies in DR had EPV ratio of 1 [58] to 9 [24, 52], which might cause overfitted model. In DN, eight [21, 22, 68, 73, 76, 77, 79, 83] and seven [27, 28, 85,86,87, 91, 92] studies might be over-optimistic with the EPV ratio less than 10 for CKD and ESRD, respectively. Overfitting may result in poorer performance in external validation compared with derived-performance. As a result, performances of the traditional statistical models (i.e. logit, Cox) were quite varied across studies. However, ML may be better particularly when predictors themselves have collinearity and high-dimensional interaction amongst predictors. With the rapid era of big data, digitalisation and modern electronic medical records may increase used of ML techniques in derived and validation model.
As the backbone of big data analysis, ML provides the new insight and valuable algorithm in which traditional statistical models are often inadequate. Likewise, using image/signal [56, 58] analysis for predicting DR, some investigators also applied classical ML (e.g. decision trees, random forest, Naïve Bayes and neural network) to predict DN [13, 70, 73]. Nonetheless, the results of ML are black boxes, which are often difficult to interpret due to its characteristics and algorithm complexities [104, 105].
Few other factors may also influence on external-validation performance, e.g. availability of predictors, sources of data (i.e. primary data collection, survey-data or administrative/hospital-claims data), outcome rate and assessment and also population characteristics. However, only about 20 studies (25% of derived models) were externally validated. We therefore strongly suggest that those derived models should be externally validated or updated models where appropriate. Then, impact analysis should next be performed to be more confident in applying in clinical practice.
Conclusions
This study was conducted to systematically review prognostic models of diabetic microvascular complications. Weaknesses and strengths of those prognostic models for each complication were described and commented. Some prognostic models for microvascular complications were good in discrimination in external validations, but in practice none of them performed clinical impact. The existing prognostic models for DR and CKD still need further external validation or update where appropriate. In addition, the new prognostic models should be derived using ML techniques to improve prognostic performance where required.
Availability of data and materials
All data included in this study were available in the supplementary information files.
Abbreviations
- BMI:
-
Body mass index
- CHARMS:
-
Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies
- CKD:
-
Chronic kidney disease
- D:
-
Derivative phase
- DN:
-
Diabetic nephropathy
- DR:
-
Diabetic retinopathy
- E:
-
External validation phase
- eGFR:
-
Estimated glomerular filtration rate
- EPV:
-
Event per variables
- ESRD:
-
End-stage renal disease
- GRS:
-
Genetic risk score
- HbA1c:
-
Glycated haemoglobin
- I:
-
Internal validation phase
- ML:
-
Machine learning
- PRISMA:
-
Preferred Reporting Items for Systematic Reviews and Meta-Analysis
PROBAST
Prediction model risk of bias assessment tool
- RCT:
-
Randomised controlled trials
- SBP:
-
Systolic blood pressure
- T2D:
-
Type 2 diabetes
References
International Diabetes Federation. IDF Diabetes Atlas Ninth Edition 2019. Brussels: International Diabetes Federation; 2019.
World Health Organization. Classification of diabetes mellitus. Geneva: World Health Organization; 2019.
Dougherty T, Heile M. Type 2 diabetes in the US managed care setting: the burden of disease and rationale for an oral glucagon-like peptide-1 receptor agonist. Am J Manag Care. 2020;26(16 Suppl):S325–s34.
Ahmad J. Management of diabetic nephropathy: recent progress and future perspective. Diabetes Metab Syndr. 2015;9(4):343–58.
American Diabetes Association. Microvascular complications and foot care: standards of medical care in diabetes-2020. Diabetes Care. 2020;43(Suppl 1):S135–s51.
Savelieff MG, Callaghan BC, Feldman EL. The emerging role of dyslipidemia in diabetic microvascular complications. Curr Opin Endocrinol Diabetes Obes. 2020;27(2):115–23.
Faselis C, Katsimardou A, Imprialos K, Deligkaris P, Kallistratos M, Dimitriadis K. Microvascular complications of type 2 diabetes mellitus. Curr Vasc Pharmacol. 2020;18(2):117–24.
Farran B, Channanath AM, Behbehani K, Thanaraj TA. Predictive models to assess risk of type 2 diabetes, hypertension and comorbidity: machine-learning algorithms and validation using national health data from Kuwait—a cohort study. BMJ Open. 2013;3(5):e002457.
American Diabetes Association. Economic costs of diabetes in the U.S. in 2017. Diabetes Care. 2018;41(5):917–28.
Seuring T, Archangelidi O, Suhrcke M. The economic costs of type 2 diabetes: a global systematic review. Pharmacoeconomics. 2015;33(8):811–31.
Aspelund T, Thornorisdottir O, Olafsdottir E, Gudmundsdottir A, Einarsdottir AB, Mehlsen J, et al. Individual risk assessment and information technology to optimise screening frequency for diabetic retinopathy. Diabetologia. 2011;54(10):2525–32.
Basu S, Sussman JB, Berkowitz SA, Hayward RA, Yudkin JS. Development and validation of risk equations for complications of type 2 diabetes (RECODe) using individual participant data from randomised trials. Lancet Diabetes Endocrinol. 2017;5(10):788–98.
Dagliati A, Marini S, Sacchi L, Cogni G, Teliti M, Tibollo V, et al. Machine learning methods to predict diabetes complications. J Diabetes Sci Technol. 2018;12(2):295–302.
Kengne AP, Gouking P, Wawo E, Koki G, Balti E, Hakapoka H, et al. Cardiovascular risk profile, diabetes specific factors, and prevalent microvascular eye complications in sub-Saharan Africans with type 2 diabetes. Int J Diabetes Developing Countries. 2015;35:349–55.
Ogunyemi O, Kermah D. Machine learning approaches for detecting diabetic retinopathy from clinical and public health records. AMIA Annual Symposium Proc AMIA Symposium. 2015;2015:983–90.
Ogunyemi O, Teklehaimanot S, Patty L, Moran E, George S. Evaluating predictive modeling’s potential to improve teleretinal screening participation in urban safety net clinics. Stud Health Technol Informatics. 2013;192:162–5.
Oh E, Yoo TK, Park EC. Diabetic retinopathy risk prediction for fundus examination using sparse learning: a cross-sectional study. BMC Medical Informatics Decision Making. 2013;13:106.
Semeraro F, Parrinello G, Cancarini A, Pasquini L, Zarra E, Cimino A, et al. Predicting the risk of diabetic retinopathy in type 2 diabetic patients. J Diabetes Complications. 2011;25(5):292–7.
Blech I, Katzenellenbogen M, Katzenellenbogen A, Wainstein J, Rubinstein A, Harman-Boehm I, et al. Predicting diabetic nephropathy using a multifactorial genetic model. PLoS One. 2011;6(4):e18743.
Dunkler D, Gao P, Lee SF, Heinze G, Clase CM, Tobe S, et al. Risk prediction for early CKD in type 2 diabetes. Clin J Am Soc Nephrol. 2015;10(8):1371–9.
Klisic A, Kavaric N, Ninic A. Retinol-binding protein 4 versus albuminuria as predictors of estimated glomerular filtration rate decline in patients with type 2 diabetes. J Res Med Sci. 2018;23:44.
Miao DD, Pan EC, Zhang Q, Sun ZM, Qin Y, Wu M. Development and validation of a model for predicting diabetic nephropathy in Chinese people. Biomed Environ Sci. 2017;30(2):106–12.
Nowak N, Skupien J, Smiles AM, Yamanouchi M, Niewczas MA, Galecki AT, et al. Markers of early progressive renal decline in type 2 diabetes suggest different implications for etiological studies and prognostic tests development. Kidney International. 2018;93(5):1198–206.
Welsh P, Woodward M, Hillis GS, Li Q, Marre M, Williams B, et al. Do cardiac biomarkers NT-proBNP and hsTnT predict microvascular events in patients with type 2 diabetes? Results from the ADVANCE trial. Diabetes Care. 2014;37(8):2202–10.
Wu M, Lu J, Zhang L, Liu F, Chen S, Han Y, et al. A non-laboratory-based risk score for predicting diabetic kidney disease in Chinese patients with type 2 diabetes. Oncotarget. 2017;8(60):102550–8.
Elley CR, Robinson T, Moyes SA, Kenealy T, Collins J, Robinson E, et al. Derivation and validation of a renal risk score for people with type 2 diabetes. Diabetes Care. 2013;36(10):3113–20.
Garlo KG, White WB, Bakris GL, Zannad F, Wilson CA, Kupfer S, et al. Kidney biomarkers and decline in eGFR in patients with type 2 diabetes. Clin J Am Soc Nephrol. 2018;13(3):398–405.
Li HY, Lin HA, Nien FJ, Wu VC, Jiang YD, Chang TJ, et al. Serum vascular adhesion protein-1 predicts end-stage renal disease in patients with type 2 diabetes. PLoS One. 2016;11(2):e0147981.
Wan EYF, Fong DYT, Fung CSC, Yu EYT, Chin WY, Chan AKC, et al. Prediction of new onset of end stage renal disease in Chinese patients with type 2 diabetes mellitus - a population-based retrospective cohort study. BMC Nephrol. 2017;18(1):257.
van der Heijden AA, Walraven I. van ‘t Riet E, Aspelund T, Lund SH, Elders P, et al. Validation of a model to estimate personalised screening frequency to monitor diabetic retinopathy. Diabetologia. 2014;57(7):1332–8.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
Debray TP, Damen JA, Snell KI, Ensor J, Hooft L, Reitsma JB, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ. 2017;356:i6460.
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.
Altman DG, Bland JM. How to obtain the confidence interval from a P value. Bmj. 2011;343:d2090.
Debray TP, Vergouwe Y, Koffijberg H, Nieboer D, Steyerberg EW, Moons KG. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol. 2015;68(3):279–89.
DerSimonian R, Laird N. Meta-analysis in clinical trials revisited. Contemp Clin Trials. 2015;45(Pt A):139–45.
Thorlund K, Wetterslev J, Awad T, Thabane L, Gluud C. Comparison of statistical inferences from the DerSimonian-Laird and alternative random-effects model meta-analyses - an empirical assessment of 920 Cochrane primary outcome meta-analyses. Res Synth Methods. 2011;2(4):238–53.
Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58(9):882–93.
Song F, Khan KS, Dinnes J, Sutton AJ. Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. Int J Epidemiol. 2002;31(1):88–95.
StataCorp. Stata Statistical Software: Release 16. College Station: StataCorp LLC; 2019.
Hippisley-Cox J, Coupland C. Development and validation of risk prediction equations to estimate future risk of blindness and lower limb amputation in patients with diabetes: cohort study. Bmj. 2015;351:h5441.
Liao WL, Lin JM, Chen WL, Hsieh MC, Wu CM, Chang YW, et al. Multilocus genetic risk score for diabetic retinopathy in the Han Chinese population of Taiwan. Scientific Reports. 2018;8(1):14535.
Yusufi FNK, Ahmed A, Ahmad J. Modelling and developing diabetic retinopathy risk scores on Indian type 2 diabetes patients. Int J Diabetes Developing Countries. 2019;39(1):29–38.
Basu S, Sussman JB, Berkowitz SA, Hayward RA, Bertoni AG, Correa A, et al. Validation of risk equations for complications of type 2 diabetes (RECODe) using individual participant data from diverse longitudinal cohorts in the U.S. Diabetes Care. 2018;41(3):586–95.
Hosseini SM, Maracy MR, Amini M, Baradaran HR. A risk score development for diabetic retinopathy screening in Isfahan-Iran. J Res Med Sci. 2009;14(2):105–10.
Soto-Pedre E, Pinies JA, Hernaez-Ortega MC. External validation of a risk assessment model to adjust the frequency of eye-screening visits in patients with diabetes mellitus. J Diabetes Complications. 2015;29(4):508–11.
Wang J, Chen H, Zhang H, Yang F, Chen RP, Li YB, et al. The performance of a diabetic retinopathy risk score for screening for diabetic retinopathy in Chinese overweight/obese patients with type 2 diabetes mellitus. Ann Med. 2014;46(6):417–23.
GarcÃa-Fiñana M, Hughes DM, Cheyne CP, Broadbent DM, Wang A, Komárek A, et al. Personalized risk-based screening for diabetic retinopathy: a multivariate approach versus the use of stratification rules. Diabetes Obes Metab. 2019;21(3):560–8.
Mo R, Shi R, Hu Y, Hu F. Nomogram-based prediction of the risk of diabetic retinopathy: a retrospective study. J Dia Res. 2020;2020. https://doi.org/10.1155/2020/7261047.
Ochs A, McGurnaghan S, Black MW, Leese GP, Philip S, Sattar N, et al. Use of personalised risk-based screening schedules to optimise workload and sojourn time in screening programmes for diabetic retinopathy: a retrospective cohort study. PLoS Med. 2019;16(10):e1002945.
Ogunyemi OI, Gandhi M, Tayek C. Predictive models for diabetic retinopathy from non-image teleretinal screening data. AMIA Jt Summits Transl Sci Proc. 2019;2019:472–7.
Tsao HY, Chan PY, Su EC. Predicting diabetic retinopathy and identifying interpretable biomedical features using machine learning algorithms. BMC Bioinformatics. 2018;19(Suppl 9):283.
Zhu X, Wu H, Ge C, Chen Y, Zhao X, Kong M, et al. Establishment and verification of a risk prediction model of diabetic retinopathy in patients with type 2 diabetes in the community. Chin Gen Pract. 2020;23(6):712–5.
Azizi-Soleiman F, Heidari-Beni M, Ambler G, Omar R, Amini M, Hosseini SM. Iranian risk model as a predictive tool for retinopathy in patients with type 2 diabetes. Can J Diabetes. 2015;39(5):358–63.
Chong YH, Fan Q, Tham YC, Gan A, Tan SP, Tan G, et al. Type 2 diabetes genetic variants and risk of diabetic retinopathy. Ophthalmology. 2017;124(3):336–42.
Han Y, Schneck ME, Bearse MA Jr, Barez S, Jacobsen CH, Jewell NP, et al. Formulation and evaluation of a predictive model to identify the sites of future diabetic retinopathy. Investigative Ophthalmol Visual Sci. 2004;45(11):4106–12.
Mendoza-Herrera K, Quezada AD, Pedroza-Tobias A, Hernandez-Alcaraz C, Fromow-Guerra J, Barquera S. A diabetic retinopathy screening tool for low-income adults in Mexico. Preventing Chronic Dis. 2017;14:E95.
Ng JS, Bearse MA Jr, Schneck ME, Barez S, Adams AJ. Local diabetic retinopathy prediction by multifocal ERG delays over 3 years. Investigative Ophthalmol Visual Sci. 2008;49(4):1622–8.
Tanaka S, Tanaka S, Iimuro S, Yamashita H, Katayama S, Akanuma Y, et al. Predicting macro- and microvascular complications in type 2 diabetes: the Japan Diabetes Complications Study/the Japanese Elderly Diabetes Intervention Trial risk engine. Diabetes Care. 2013;36(5):1193–9.
Bresnick GH, Palta M. Predicting progression to severe proliferative diabetic retinopathy. Arch Ophthalmol. 1987;105(6):810–4.
Cichosz SL, Johansen MD, Knudsen ST, Hansen TK, Hejlesen O. A classification model for predicting eye disease in newly diagnosed people with type 2 diabetes. Diabetes Res Clin Practice. 2015;108(2):210–5.
Mogilevskyy. Predicting the development of diabetic retinopathy based on identifcation of rs759853 and rs9640883 in the AKR1B1 gene. J Ophthalmol 2017;IV:477.
Cichosz SL, Johansen MD, Hejlesen O. Toward big data analytics: review of predictive models in management of diabetes and its complications. J Diabetes Sci Technol. 2015;10(1):27–34.
Ganjifrockwala FA, Joseph JT, George G. Evaluation of kidney function and risk factors of retinopathy in type 2 diabetes mellitus people in South Africa. Diabetes Res Clin Pract. 2017;127:218–23.
Hu Y, Shi R, Mo R, Hu F. Nomogram for the prediction of diabetic nephropathy risk among patients with type 2 diabetes mellitus based on a questionnaire and biochemical indicators: a retrospective study. Aging (Albany NY). 2020;12(11):10317–36.
Jiang S, Fang J, Yu T, Liu L, Zou G, Gao H, et al. Novel model predicts diabetic nephropathy in type 2 diabetes. Am J Nephrol. 2020;51(2):130–8.
Liao LN, Li TC, Li CI, Liu CS, Lin WY, Lin CH, et al. Genetic risk score for risk prediction of diabetic nephropathy in Han Chinese type 2 diabetes patients. Scientific Reports. 2019;9(1):19897.
Peters KE, Davis WA, Ito J, Bringans SD, Lipscombe RJ, Davis TME. Validation of a protein biomarker test for predicting renal decline in type 2 diabetes: the Fremantle diabetes study phase II. J Diabetes Complications. 2019;33(12):107406.
Qian YS, Ming MF. Predicting the risk of chronic kidney disease among type 2 diabetes mellitus patients in a primary care setting: an evaluation of the qkidney model. Malays J Med Health Sci. 2019;15(3):67–73.
Rodriguez-Romero V, Bergstrom RF, Decker BS, Lahu G, Vakilynejad M, Bies RR. Prediction of nephropathy in type 2 diabetes: an analysis of the ACCORD trial applying machine learning techniques. Clin Transl Sci. 2019;12(5):519–28.
Song X, Waitman LR, Hu Y, Yu ASL, Robbins DC, Liu M. Robust clinical marker identification for diabetic kidney disease with ensemble feature selection. J Am Med Informatics Assoc. 2019;26(3):242–53.
Wysham CH, Gauthier-Loiselle M, Bailey RA, Manceur AM, Lefebvre P, Greenberg M, et al. Development of risk models for major adverse chronic renal outcomes among patients with type 2 diabetes mellitus using insurance claims: a retrospective observational study. Curr Med Res Opin. 2020;36(2):219–27.
Goldfarb-Rumyantzev AS, Pappas L. Prediction of renal insufficiency in Pima Indians with nephropathy of type 2 diabetes mellitus. Am J Kidney Dis. 2002;40(2):252–64.
Jardine MJ, Hata J, Woodward M, Perkovic V, Ninomiya T, Arima H, et al. Prediction of kidney-related outcomes in patients with type 2 diabetes. Am J Kidney Dis. 2012;60(5):770–8.
Jenks SJ, Conway BR, McLachlan S, Teoh WL, Williamson RM, Webb DJ, et al. Cardiovascular disease biomarkers are associated with declining renal function in type 2 diabetes. Diabetologia. 2017;60(8):1400–8.
Kim JH, Oh SY, Kim EH, Lee MJ, Jeon YK, Kim BH, et al. Addition of nonalbumin proteinuria to albuminuria improves prediction of type 2 diabetic nephropathy progression. Diabetol Metabolic Syndrome. 2017;9:68.
Lindhardt M, Persson F, Zurbig P, Stalmach A, Mischak H, de Zeeuw D, et al. Urinary proteomics predict onset of microalbuminuria in normoalbuminuric type 2 diabetic patients, a sub-study of the DIRECT-Protect 2 study. Nephrol Dialysis Transplantation. 2017;32(11):1866–73.
Low S, Lim SC, Zhang X, Zhou S, Yeoh LY, Liu YL, et al. Development and validation of a predictive model for chronic kidney disease progression in type 2 diabetes mellitus based on a 13-year study in Singapore. Diabetes Res Clin Pract. 2017;123:49–54.
Peters KE, Davis WA, Ito J, Winfield K, Stoll T, Bringans SD, et al. Identification of novel circulating biomarkers predicting rapid decline in renal function in type 2 diabetes: the Fremantle diabetes study phase II. Diabetes Care. 2017;40(11):1548–55.
Riphagen IJ, Kleefstra N, Drion I, Alkhalaf A, van Diepen M, Cao Q, et al. Comparison of methods for renal risk prediction in patients with type 2 diabetes (ZODIAC-36). PLoS One. 2015;10(3):e0120477.
Saulnier PJ, Gand E, Velho G, Mohammedi K, Zaoui P, Fraty M, et al. Association of circulating biomarkers (adrenomedullin, TNFR1, and NT-proBNP) with renal function decline in patients with type 2 diabetes: a French prospective cohort. Diabetes Care. 2017;40(3):367–74.
Yang JK, Wang YY, Liu C, Shi TT, Lu J, Cao X, et al. Urine proteome specific for eye damage can predict kidney damage in patients with type 2 diabetes: a case-control and a 5.3-year prospective cohort study. Diabetes care. 2017;40(2):253–60.
Zobel EH, von Scholten BJ, Reinhard H, Persson F, Teerlink T, Hansen TW, et al. Symmetric and asymmetric dimethylarginine as risk markers of cardiovascular disease, all-cause mortality and deterioration in kidney function in persons with type 2 diabetes and microalbuminuria. Cardiovasc Diabetol. 2017;16(1):88.
Lin CC, Li CI, Liu CS, Lin WY, Lin CH, Yang SY, et al. Development and validation of a risk prediction model for end-stage renal disease in patients with type 2 diabetes. Sci Rep. 2017;7(1):10177.
Yang XL, So WY, Kong AP, Clarke P, Ho CS, Lam CW, et al. End-stage renal disease risk equations for Hong Kong Chinese patients with type 2 diabetes: Hong Kong Diabetes Registry. Diabetologia. 2006;49(10):2299–308.
Foster MC, Inker LA, Hsu CY, Eckfeldt JH, Levey AS, Pavkov ME, et al. Filtration markers as predictors of ESRD and mortality in Southwestern American Indians with type 2 diabetes. Am J Kidney Dis. 2015;66(1):75–83.
Fufaa GD, Weil EJ, Nelson RG, Hanson RL, Bonventre JV, Sabbisetti V, et al. Association of urinary KIM-1, L-FABP, NAG and NGAL with incident end-stage renal disease and mortality in American Indians with type 2 diabetes mellitus. Diabetologia. 2015;58(1):188–98.
Lambers Heerspink HJ, Gansevoort RT, Brenner BM, Cooper ME, Parving HH, Shahinfar S, et al. Comparison of different measures of urinary protein excretion for prediction of renal events. J Am Soc Nephrol. 2010;21(8):1355–60.
Mise K, Hoshino J, Ueno T, Hazue R, Hasegawa J, Sekine A, et al. Prognostic value of tubulointerstitial lesions, urinary N-acetyl-beta-d-glucosaminidase, and urinary beta2-microglobulin in patients with type 2 diabetes and biopsy-proven diabetic nephropathy. Clin J Am Soc Nephrol. 2016;11(4):593–601.
Mise K, Yamaguchi Y, Hoshino J, Ueno T, Sekine A, Sumida K, et al. Paratubular basement membrane insudative lesions predict renal prognosis in patients with type 2 diabetes and biopsy-proven diabetic nephropathy. PLoS One. 2017;12(8):e0183190.
Niewczas MA, Gohda T, Skupien J, Smiles AM, Walker WH, Rosetti F, et al. Circulating TNF receptors 1 and 2 predict ESRD in type 2 diabetes. J Am Soc Nephrol. 2012;23(3):507–15.
Pavkov ME, Knowler WC, Hanson RL, Williams DE, Lemley KV, Myers BD, et al. Comparison of serum cystatin C, serum creatinine, measured GFR, and estimated GFR to assess the risk of kidney failure in American Indians with diabetic nephropathy. Am J Kidney Dis. 2013;62(1):33–41.
Keane WF, Zhang Z, Lyle PA, Cooper ME, de Zeeuw D, Grunfeld JP, et al. Risk scores for predicting outcomes in patients with type 2 diabetes and nephropathy: the RENAAL study. Clin J Am Soc Nephrol. 2006;1(4):761–7.
Sullivan LM, Massaro JM, D’Agostino RB Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med. 2004;23(10):1631–60.
Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. 2017;318(14):1377–84.
Moons KG, Kengne AP, Woodward M, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart. 2012;98(9):683–90.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Bmj. 2015;350:g7594.
Jakobsen JC, Gluud C, Wetterslev J, Winkel P. When and how should multiple imputation be used for handling missing data in randomised clinical trials - a practical guide with flowcharts. BMC Med Res Methodol. 2017;17(1):162.
Lagakos SW. Effects of mismodelling and mismeasuring explanatory variables on tests of their association with a response variable. Stat Med. 1988;7(1-2):257–74.
Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006;25(1):127–41.
Subramanian V, Mascha EJ, Kattan MW. Developing a clinical prediction score: comparing prediction accuracy of integer scores to statistical regression models. Anesth Analg. 2021;132(6):1603–13. https://doi.org/10.1213/ANE.0000000000005362.
Núñez E, Steyerberg EW, Núñez J. Regression modeling strategies. Revista Española de CardiologÃa (English Edition). 2011;64(6):501–7.
Concato J, Peduzzi P, Holford TR, Feinstein AR. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol. 1995;48(12):1495–501.
Luo W, Phung D, Tran T, Gupta S, Rana S, Karmakar C, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res. 2016;18(12):e323.
Stevens LM, Mortazavi BJ, Deo RC, Curtis L, Kao DP. Recommendations for reporting machine learning analyses in clinical research. Circ Cardiovasc Qual Outcomes. 2020;13(10):e006556.
Acknowledgements
This study was an important part of Sigit Ari Saputro’s training in PhD (Data Science for Healthcare), Faculty of Medicine Ramathibodi Hospital, Mahidol University, Thailand. The authors would like to thank Stephen John Pinder for proofreading and conducting comprehensive English language review.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors.
Author information
Authors and Affiliations
Contributions
SAS, OP and AT conceptualised the study questions and developed protocol and search strategy for the systematic review. SAS conducted the review (screening, data extraction, analyse the data) and wrote the manuscript. SAS and SK did the risk of bias assessment and wrote interpretation. AT, OP and AP edited and revised the manuscript. All authors have read and approved the final version of the manuscript to be published.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable
Consent for publication
All authors give their consent for publication.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1:
Search strategy & results from PubMed database.
Additional file 2:
Search strategy & results from Scopus database.
Additional file 3: Table S1.
Characteristics of studies included in systematic review. Table S2. Describe discrimination and calibration performances of prognostic models. Table S3. Risk of bias assessment using PROBAST. Table S4. Prognostic equations that were externally validated. Table S5. Summary of pooled C-statistic of prognostic model. Table S6. Describe variables that were included in the derivative equations.
Additional file 4: Figure S1.
PRISMA diagram for diabetic retinopathy. Figure S2. PRISMA diagram for diabetic nephropathy. Figure S3. Risk-of-bias using PROBAST. Figure S4. Overview top 20 predictors included in derived models. Figure S5. Funnel plot (C-statistics) for DR in external validation. Figure S6. Funnel plot (C-statistics) for CKD in external validation. Figure S7. Funnel plot (C-statistics) for ESRD in external validation.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Saputro, S.A., Pattanaprateep, O., Pattanateepapon, A. et al. Prognostic models of diabetic microvascular complications: a systematic review and meta-analysis. Syst Rev 10, 288 (2021). https://doi.org/10.1186/s13643-021-01841-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13643-021-01841-z
Keywords
- Meta-analysis
- Microvascular complications
- Prognostic model
- Type 2 diabetes