- Open Access
Evaluating measurement properties of subjective cognitive decline self-reported outcome measures: a systematic review
Systematic Reviews volume 11, Article number: 144 (2022)
Subjective cognitive decline (SCD) is present in the early stage of preclinical Alzheimer’s disease (AD) and is associated with an increased risk of further cognitive decline and AD dementia later in life. Early detection of at-risk groups with subjective complaints is critical for targeted dementia prevention at the earliest. Accurate assessment of SCD is crucial. However, current measures lack important psychometric evaluations and or reporting.
To systematically evaluate measurement properties of self-reported outcome measures (PROMs) used to assess SCD in the older adult population with or at risk of AD.
Methods and analysis
We used the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols 2015 Checklist for reporting. We conducted a literature search, screened, and included validation studies of SCD based on self-reported questionnaires from both population-based and clinical studies, conducted in older adults (≥ 55). We critically appraised the included primary studies using the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) guidelines.
Sixteen studies met the inclusion criteria. The included studies reported psychometric properties of 17 SCD self-reported questionnaires. We extracted data on the structural validity, internal consistency, test-retest reliability, and cross-cultural validity and found a widespread proneness to bias across studies, and a marked heterogeneity is assessed and reported measurement properties that prevented the consolidation of results.
Our findings suggest that available SCD questionnaires lack content validity evaluation. Currently available measurements of SCD lack development and validation standards. Further work is needed to develop and validate SCD self-reported measurement with good quality measurement properties.
Targeted dementia prevention requires the early detection and diagnosis of at-risk individuals of Alzheimer’s disease (AD) dementia. Several biomarkers (e.g., amyloid plaques and neurofibrillary tangles) are present in the brain many years before dementia develops . Furthermore, neuronal damage and loss may already occur in earlier stages such as mild cognitive impairment (MCI), leading to irreversible cognitive dysfunction . The US National Institute on Aging – Alzheimer’s Association (NIA-AA) proposed a preclinical stage of AD characterized by normal cognitive performance in standardized neuropsychological tests and the presence of AD biomarkers [3,4,5,6]. This stage may be accompanied by a subtle cognitive decline that is only perceived subjectively but not captured by standardized tests . Therefore, detection at this stage is of particular interest for AD prevention, including disease-modifying trials [7, 8].
Subjective cognitive decline (SCD) is defined as a self-perceived, progressive decline in cognitive abilities such as memory, executive functions, or language . Previous evidence suggests that older adults with SCD have an increased risk of further cognitive decline and conversion to MCI or AD dementia in late life [9,10,11,12]. SCD as a separate construct has been gaining more attention and is being suggested to be one of the earliest symptoms in the preclinical stage of AD [7, 13]. The Subjective Cognitive Decline Initiative (SCD-I) working group defined key concepts of SCD and propose it as a symptomatic stage of preclinical AD . However, assessment of SCD varies greatly between studies, and standardization and operationalization are lacking . Recommendations from the SCD-I working group include a thorough evaluation of psychometric properties of available self-reported measures used in the current literature . SCD is commonly assessed using a single-item approach (e.g., “Do you feel that your memory is getting worse?”) with a dichotomous response option (yes/no) [15,16,17]. This approach does not cover two important aspects of SCD. First, self-perceived decline can affect other cognitive domains than memory. By asking about memory alone, potential perceived failures in other domains (e.g., executive function or language) may be overlooked. Second, it is important to ask about the time of onset and graduality of the subjective decline. A gradual perceived change in cognitive function over recent years is more likely to be an early manifestation of AD than the persistent feeling of change that has been present for many years [7, 18, 19]. Another aspect to consider is the inclusion of an informant’s report. The proposed diagnostic criteria for SCD by the SCD-I recommends including an informant’s report of cognitive changes of the older adult. While several SCD PROMs are currently used in research and clinical studies, the methodological quality of these measures varies greatly . The importance of reliability and validity, two of the core psychometric properties in scale development, lies in the ability to capture subjective decline in cognitive functions accurately and consistently. To the best of our knowledge, our study is the first systematic review to evaluate the psychometric properties of available self-reported questionnaire that are used to assess SCD in older adults.
Building on the recommendation of the SCD-I, the main aim of this work was to conduct a systematic review to evaluate the psychometric properties of self-reported output measures used to assess SCD in older adults with or at risk of AD. The research question follows the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) format [20,21,22]:
The construct or the name(s) of the outcome measurement instrument(s) of interest: SCD in AD
The target population: older adults 55 years old and above
The type of measurement instrument of interest: self-reported questionnaires used to assess SCD in older adults in the context of AD
The measurement properties on which the review focuses: structural validity, internal consistency, test-retest reliability, and cross-cultural validity
The protocol of this systematic review was registered on PROSPERO (CRD 42020166905). The protocol and the systematic review were reported following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [23, 24].
We attempted to identify original studies that reported PROM development to assess SCD in older adults in the context of Alzheimer’s disease. We included studies that performed and reported validation of psychometric properties, specifically on the validity and reliability of SCD PROMs. We included community-based studies as well as studies conducted in memory clinics or research settings. We excluded studies that used SCD PROMs to recruit participants for specific studies or studies that did not aim to validate PROMs. We also excluded studies that developed and validated SCD PROMs for the purpose of screening or diagnosing SCD in other diseases (e.g., depression).
We included studies with older adults (55 years and older) in studies of Alzheimer’s disease (including studies about Mild Cognitive Impairment and AD dementia studies). We attempted to include studies that tested SCD PROMs in cognitively healthy adults as well as adults diagnosed with SCD, MCI, or AD dementia. We excluded studies with older adults with SCD due to any other specific, previously identified conditions such as stroke, neuropsychiatric conditions (i.e., mood disorders, psychotic disorders), trauma, delirium, or disability.
We searched for published studies using the main and most relevant biomedical databases to our study focus:
For gray literature, we used the OpenGrey database. To look for thesis and dissertations, we used Open Access Theses and Dissertations and WorldCat databases. For conference proceedings and abstracts, we used Web of Science and Scopus. Studies included in the review were not limited to a certain geographical location.
Our search strategy included iterations of the concepts Subjective Cognitive Decline AND Preclinical Alzheimer’s disease AND self-reported questionnaire AND measurement properties. The full search strategy is available in Additional file 1.
Data management: We stored all records, articles, and related material using OneDrive. We used Zotero for bibliographic management for all retrieved studies and to remove duplicates.
Selection process: Three independent reviewers (AI, JR, and MS) screened the study titles and abstracts against the eligibility criteria. The first reviewer (AI) then imported the selected studies to Rayyan—a web-based software for the title and abstract screening in systematic reviews —for the title and abstract screening phase. Any conflict between the three reviewers was resolved through discussion. Full texts of the included records were independently reviewed by the three reviewers to determine the eligibility for data extraction and analysis. This was followed by another session to resolve discrepancies between the three reviewers. In case of unresolved conflict, a senior researcher (EA) was consulted to make the final decision.
Data collection process: The first reviewer (AI) abstracted the data from full records independently using the COSMIN data abstraction tables. When full articles were not available, we contacted the corresponding authors and requested the full text. The three reviewers (AI, JR, and MS) completed the risk of bias checklist and rating of the quality of measurements using the COSMIN material.
We used the COSMIN methodology in a modular manner to evaluate any reported psychometric property in the included study. However, for the purpose of this systematic review, we will focus on the internal structure of PROMs and qualitatively analyze the structural validity, reliability (internal consistency and test-retest reliability), and cross-cultural validity. The main outcomes are the following measurement properties:
Self-reported outcome measurement development
Content validity evaluation
Internal consistency and test-retest reliability
Because there is no gold standard measurement to assess SCD, we ignored the results on criterion validity in the included studies and did not include them in the analysis. Furthermore, a number of the included studies also reported construct validity (convergent and concurrent validity) of the SCD PROM, however with clinically administered assessment measures of cognitive decline (e.g., MMSE). We ignored these results as well because construct validity should be tested with other validated measures that assess SCD.
Data extraction was conducted independently by three reviewers. We used the developed extraction sheet by COSMIN and piloted it at the start of data extraction. We extracted the following information variables:
The characteristics of the self-reported outcome measurement include the name of the measure, a reference to the article in which the development of the measure is described, constructs being measured, language and study population for which the measure was developed, intended context of use, the available language version of the measure, number of items in each scale, and response options.
Characteristics of the included samples include geographical location, language, target population, sample size, percentage of female participants, and mean age of the study sample.
Methodological quality ratings per measurement property per PROM.
Measures of effect
Each measurement property is evaluated by rating the relevant sub-items listed below. For example, in internal consistency, if the validation study of the PROM reports that Cronbach’s alpha was calculated for continuous scores, then this sub-item is rated “very good.” If not, it is rated “doubtful” or “inadequate” depending on which other statistical tests were performed. The measures of effects per psychometric property are presented in Table 1.
Risk of bias
We evaluated the risk of bias using the COSMIN Risk of Bias Checklist [20,21,22]. The checklist assesses the quality of the relevant main outcomes described above. Each psychometric property is evaluated by scoring a set of items about the conduction and reporting of the specific property. Each item is scored either “V” (very good), “A” (adequate), “D” (doubtful), or “I” (inadequate) according to the instruction of the COSMIN rating guideline. The total score for each psychometric property is given based on the “worst score count” method. The COSMIN guidelines instruct the raters to rate the overall quality of a property by taking the lowest rating given to any of the sub-items. The risk of bias per psychometric property per PROM was evaluated by the three reviewers (AI, JR, and MS). Any discrepancy between the three reviewers was discussed to reach a consensus.
Strategy for data synthesis
We did not conduct a meta-analysis, and we only described the quality of psychometric properties testing in the selected studies. We used the COSMIN criteria to evaluate each psychometric property that was tested and reported for each SCD PROM. We evaluated validity (structural validity) and reliability (internal consistency and test-retest reliability) properties. We further evaluated cross-cultural validity and convergent validity when possible. We evaluated each reported psychometric property per PROM before judging its risk of bias. Based on the statistical analysis and reported results of each psychometric property, we also provided a qualitative assessment on whether the results are “sufficient,” “indeterminate,” or “insufficient.”
Analysis of subgroups or subsets
We did not conduct an analysis of subgroups or subsets in this review.
Confidence in cumulative evidence
The COSMIN guidelines recommend using the modified version of the GRADE approach (Grading of Recommendations Assessment, Development and Evaluation) to grade the strength and quality of the collected evidence. Because the identified studies did not have content validity studies, we were not able to grade the selected SCD PROM studies using the modified GRADE approach.
Results of the search
We completed the search on 10 June 2021. We identified 364 records in the initial database search. We further identified 13 records through hand-searching the references of some of the relevant studies. After duplicate removal, there were 290 records remaining. The three reviewers (AI, JR, and MS) screened the title and abstract of the 290 records using Rayyan software. After the completion of the screening phase, the inter-rater reliability and agreement between the three reviewers were measured using Fleiss kappa  and agreement percentage in RStudio . The results indicate substantial agreement between the reviewers (Fleiss kappa = 0.662, p-value < 0.0001). Moreover, the percentage agreement with zero tolerance was 78.4%. After the resolution of the conflict between the three reviewers, we excluded 208 records that did not meet the inclusion criteria and 10 records that did not have the full text available (conference proceedings). One further duplicate study was identified at the stage of full-text review. We screened the full texts of the remaining 71 records.
The final number of included studies was sixteen studies. The search and selection of studies are demonstrated in a PRISMA flow chart (Fig. 1). All studies developed and validated SCD PROMS in high-income settings, mainly in the USA (37.5%) and Europe (56.25%). Only one study was conducted in Asia (South Korea). Table 2 summarizes the characteristics of the included studies that reported measurement properties of SCD self-reported questionnaires in older adults. Additional file 1: Table S3 provides an overview of excluded studies after full-text review, with an explanation on the reasons to exclude as well as citations (available in the supplementary material).
Fifteen studies validated the questionnaires in a sample including both cognitively healthy older adults as well as adults diagnosed with SCD, MCI, or AD. Seven included studies developed and validated SCD questionnaires only in cognitively healthy older adults [30, 32, 36, 38, 39, 43, 44]. One study included participants with MCI only . Older adults with MCI were further included in 6 of the included studies [31, 33, 35, 37, 40, 41]. Only two studies included participants who received a prior diagnosis of SCD . Participants with clinically diagnosed AD dementia were included in two studies [40, 45].
Assessed cognitive domains
Sixteen of the seventeen SCD PROMs included items to assess subjective decline in memory function. Nine PROMs looked only at the memory and included different aspects of memory aspects such as facial recognition, spatial topographic memory, word and fact recall/semantic memory, general forgetfulness, everyday task-oriented memory, numeric recall, and remote personal memory . Other PROMs focusing on memory aimed to assess retrospective and prospective memory [37, 39] and episodic memory  or to capture self-perceived change in memory functioning overall . One PROM was developed to assess subjective decline in spatial navigation skills only . Six PROMs assess subjective complaints in executive functions and praxis [31, 32, 37, 38, 40,41,42]. Self-perceived decline in language abilities was assessed only in three PROMs [37, 40,41,42]. Lastly, social cognition was assessed in one PROM .
Reported psychometric properties
As shown in Table 2, of the included studies, 81.25% performed and reported aspects of the PROM development procedure (n = 13) [30,31,32,33,34,35,36, 39, 40, 43, 45] 75% (n = 12) of which reported conducting a cognitive interview or piloting the generated pool of questionnaire items to assess comprehensibility and comprehensiveness of the selected items [30,31,32,33,34,35,36,37,38, 40, 43, 45]. However, all studies but one  did not report details regarding the piloting phase. None of the included studies tested for content validity of the developed PROM. On the other hand, the majority of studies tested and reported structural validity (75%, n = 12) for thirteen PROMs [30, 31, 35,36,37,38,39,40, 42,43,44,45]. .Four of which (25%) used exploratory factor analysis (EFA) only [30, 31, 37, 42], while two studies used EFA followed by confirmatory factor analysis (CFA) [35, 36]. Four studies only reported and/or conducted CFA only [38, 39, 44, 45]. Internal consistency using Cronbach’s alpha was reported in 68.75% of studies (n = 11) for 13 PROMs [30, 32,33,34, 36, 38,39,40,41, 43,44,45]. One study used item response theory (IRT) to test for internal consistency and reported the standard error (SE θ) . One study reported that internal consistency was tested; however, no statistic was provided . Test-retest reliability was reported in four studies [30, 33, 36, 45]. Only four studies (25%) indicate cross-cultural validation of SCD PROMs in other languages [31, 32, 37, 39]. Two studies evaluated and reported convergent validity with other SCD PROMs [33, 43].
Risk of bias
All twelve studies that reported PROM development steps received “I” rating (i.e., high risk of bias) due to the lack of conducting or reporting a cognitive interview or asking about the comprehensibility or comprehensiveness of the PROM items. Seven of the thirteen PROMs performed CFA and received a “V” rating in structural validity testing, therefore had a low risk of bias [36, 38,39,40, 44, 45]. One study was rated “A” for structural validity and was judged to have a moderate risk of bias . The remaining five PROMs were judged to have a high risk of bias due to having an inadequate sample size and receiving “I” rating on the reported structural validity [30, 35, 38, 42, 43].
Regarding internal consistency, of the 15 PROMs that were tested for internal consistency studies, 12 were judged to have a low risk of bias (“V” rating) [34,35,36, 38,39,40,41, 43,44,45,46]. One PROM, the spatial navigation questionnaire was judged to have a moderate risk of bias (“A” rating) . The EMQ and the CFI both had a D rating, i.e., show a high risk of bias [31, 32]. The remaining PROMs were judged to have a high risk of bias as well (“I” rating) [37, 42].
Moreover, two PROMs, the spatial navigation questionnaire and the MFQ [30, 36], had a V rating for the test-rest reliability and were therefore of low risk of bias. On the other hand, the MAC-Q and the SMCQ were of high risk of bias (“D” rating) [33, 45]. All four PROMs that evaluated cross-cultural validity of the translated version of the questionnaire were of low risk of bias (“V” rating) [31, 37, 39]. The risk of bias evaluation is available in Additional file 1: Table S1.
Included SCD PROMS and the quality of the reported psychometric properties
The included studies evaluated 17 PROMs in total that are used to assess SCD in adults. Table 3 provides details of the validated SCD PROMS, which cognitive domains are covered, and the response options and their scoring system. The results of the reported psychometric property per PROM as well as a summary of the quality of each measurement property of the abovementioned SCD PROMs, as rated against the COSMIN criteria of good measurement property, are available in the appendix (in Additional file 1: Tables S2–S3). The focus of each PROM is elaborated below.
The spatial navigation questionnaire is a self-report questionnaire that assesses self-perceived decline in spatial navigation skills over the past several years . It includes self- and informant reports, and each part consists of 20 items of statements regarding the change in navigation abilities. The response options range from 1 (strongly disagree) to 7 (strongly agree). The questionnaire was validated in a laboratory setting with a sample of 91 cognitively healthy older adults (mean age = 69, SD = 6). The questionnaire was administered to participants and their informants in two visits, 3 months apart. The authors report using exploratory factor analysis (EFA) to identify 20 out of 30 initial items that measure self-perceived change in navigation skills. However, due to deploying EFA only, the quality of the structural validity was deemed “indeterminate.” Internal consistency for the participant’s form was sufficient (Cronbach’s α = 0.965, CI = 0.953–0.974), and for the informant’s form (Cronbach’s α = 0.957, CI = 0.942–0.970) as well as test-rest reliability for the participants (ICC = 0.838, CI = 0.743–0.900) and the informant part (ICC = 0.723, CI = 0.552–0.835) .
Everyday Memory Questionnaire (EMQ) is a 25-item questionnaire that comprises a five-factor structure : (1) forgetfulness of immediate information (FII) was associated with fails in immediate retrieval as well as naming impairment; (2) executive functions; (3) prospective memory (PM); (4) forgetfulness of common objects (FCO); and (5) spatial orientation (SO). The PROM was validated in a sample of 844 participants, 766 of which were cognitively healthy controls (90.8%) and 78 had MCIs (9.2%). The mean age for cognitively healthy participants was 74.07 (SD = 3.80) and 76.08 (SD = 4.06) for participants with MCI. Items were scored on a 3-point scale, with 0 representing “never, rarely,” 1 “occasionally, sometimes,” and 2 “frequently, almost always.” The total score ranged from 0 to 56. The authors report testing for structural validity (EFA) and internal consistency. However, no statistical test for internal consistency was reported. Cross-cultural validity is implicitly indicated because the EMQ was translated from English to Spanish and was validated in a Spanish-speaking population.
The cognitive functions instrument (CFI) contains 14 questions and has two parts, a participant and an informant’s form . It assesses the presence of subjective cognitive concerns in older adults. The response options are yes = 1, maybe = 0.5, and no = 0, and the total score of the questionnaire ranges from 0 to 14. The CFI was validated in the Italian language in a sample of (mean age = 70.9, SD = 5.1). The authors report a more detailed and thorough translation and cultural adaptation process which indicated sufficient cross-cultural validity. The CCFI was only evaluated for the internal consistency for the participant’s (Cronbach’s α = 0.77, 95% CI = 0.72–0.83) as well as the informant’s part (group 1: Cronbach’s α = 0.77, 95% CI 0.70–0.85; group 2: Cronbach’s α = 0.72, 95% CI = 0.66 to 0.78) .
The Memory Complaint Questionnaire (MAC-Q) is a 6-item self-report questionnaire that assesses age-associated memory decline in older adults in a clinical trial for experimental treatment of age-associated cognitive decline . The response options are either yes or no, and the total score ranges from 7 to 35. The authors reported that MAC-Q was administered again after 12 weeks to assess test-retest reliability; however, the reported statistic was not specified. The reported internal consistency of the MAC-Q was not sufficient (Cronbach’s α = 0.57). The study also evaluated the convergent validity of another validated SCD questionnaire, the MAC-S. Note that the authors referred to this as “concurrent validity.” The Memory Assessment Clinics-Self-rating (MAC-S) is a 49-item memory questionnaire . Twenty-one items assess the person’s ability to remember specific types of information, and 24 items evaluate how often specific memory problems are experienced. The last 4 items ask about the person’s overall assessment of his or her memory. The MAC-S assesses different memory skills (e.g., facial recognition, spatial topographic memory, word and fact recall). The response options are rated on a 5-point Likert scale and the total score of the 49 items. The reported concurrent validity was sufficient (r = 0.41, p-value < 0.001).
One study reported measurement properties of two different PROMs. The first was a set of six items extracted from the Attitude Toward Intellectual Aging Scale of the Personality in Intellectual Aging Contexts (PIC) Inventory . The response options ranged from 1 (strongly agree) to 6 (strongly disagree) and were summed for the overall score. The second PROM is the General Frequency of Forgetting Scale of the Memory Functioning Questionnaire (MFQ) . The subscale of the MFQ has 14 items that assesses the change in memory. Response options ranged from 1 (always) to 7 (never) and were summed to obtain the overall score. For both scales, higher scores reflected a greater perception of cognitive decline. The authors report sufficient internal consistency for both scales (Cronbach α = 0.81 and 0.92, respectively).
Gifford et al.  developed a scale of 9 items to assess SCD using EFA followed by confirmatory factor analysis (CFA) to evaluate the unidimensionality of the scale, as well as item-response theory (IRT) to test for internal consistency. Participants were mailed a questionnaire containing 57 items that ask about everyday memory failures. In the final version, the response options were dichotomous (yes/no) for the first 6 questions. For the remaining questions, the responses were rated on a 3-point Likert scale, ranging from “always,” “sometimes,” or “never a problem.” The final version of the 9-item questionnaire shows sufficient structural validity (CFA was performed) and internal consistency (the mean SE (θ) score was 20.12–0.90 for cognitively healthy participants and 0.34–0.83 for participants with MCI).
The Memory Functioning Questionnaire (MFQ) was developed and validated in 778 cognitively healthy older adults . It contains 64 items that assess subjective memory worsening. The items are rated on a 7-point Likert scale that ranges from. Structural validity was tested by EFA followed by CFA. The final version of 64 items consists of four factors: (1) general frequency of forgetting, (2) seriousness of forgetting, (3) retrospective functioning, and (4) mnemonics usage. Reported Cronbach α for internal consistency of each factor was 0.94, 0.94, 0.89, and 0.83, respectively. The authors refer to test-retest reliability being measured; however, no ICC or weighted kappa was calculated, and therefore, test-retest reliability was rated “insufficient.”
The Cognitive Difficulties Scale (CDS) is a 39-item questionnaire that assesses how often someone is currently experiencing cognitive difficulties in everyday life [37, 48]. The items are rated on a 5-point scale, ranging from “never” = 0 to “very often” = 4. The authors of the included study validated the SCD in the French language in a sample of 185 older adults (mean age: cognitively healthy = 69, SD = 7.2; MCI = 73, SD = 7.2; SCD = 68, SD = 7.3). The reported structural validity assessment was rated “inadequate” due to the sample size.
The complainer profile identification (CPI) is a 17-item scale that assesses subjective change in memory, attention, and executive function . It was validated in a sample of 734 German adults, 83.67 (11.4%) of which were above 65 years old (mean age = 43.15, SD = 17.17). The items are rated on a 5-points Likert scale, ranging from “never” = 0 to “very often” = 4. Both structural validity (CFA) and internal consistency (Cronbach’s α = 0.87) were sufficient as rated by the criteria COSMIN of good measurement property.
Papaliagkas et al.  reported structural validity, reliability, and cross-cultural validity of the Greek versions of the Cognitive Failures Questionnaire (CFQ)  and the Prospective and Retrospective Memory Questionnaire (PRMQ) . Both scales were translated and validated in a sample of 295 Greek older adults (older adults mean age = 69.9, SD = 3.6; older-old adults mean age = 83.5, SD = 3.3). The CFQ shows sufficient structural validity (CFA: χ2 (106, N = 449) = 260.46, p = 0.011, CFI = 0.985, SRMR = 0.036, RMSEA = 0.023) as well as internal consistency (Cronbach’s α = 0.93). On the other hand, the PRMQ also shows sufficient structural validity (CFA: χ2 (88, N = 464) = 186.14, p < 0.001, CFI = 0.959, SRMR = 0.035, RMSEA = 0.049) and internal consistency (Cronbach’s α = 0.84, for self-rated memory; α = 0.84, for self-rated prospective memory; and α = 0.79, for self-rated retrospective memory).
The Subjective Cognitive Decline Questionnaire (SCD-Q) is a 24-item scale that was validated in a cohort of 794 Spanish speakers to assess self-perceived change in several cognitive domains in older adults over the preceding 2 years . The scale has a self and informant parts, each consisting of the same 24 items. The response options are dichotomous (yes/no) and the total score for each part of the questionnaire ranged from 0 to 24 points. The authors report sufficient internal consistency for the self (Cronbach’s α = 0.90) and informant parts (Cronbach’s α = 0.93). Structural validity of the SCD-Q was investigated in the original study as well as another included study conducted in Spain, both using EFA [40, 42], and therefore, both studies were judged to have “indeterminate” quality of structural validity.
The Cognitive Change Index (CCI) is a 20-item scale, developed and validated in a sample of 267 older adults in the USA . It has a self and informant’s parts consisting of the same questions. The items are rated on a 5-point Likert scale ranging from 1 = no change or normal ability, 2 = minimal change or slight/occasional problem, 3 = some change or mild problem, 4 = clearly noticeable change or moderate problem, to 5 = much worse or severe problem. The authors report sufficient internal consistency (Cronbach α: self = 0.96, informant = 0.98). No structural validity analysis was reported. Therefore, internal consistency could not be rated or considered sufficient.
The Cognitive Dysfunction Questionnaire (CDQ) is a 20-item questionnaire that was validated in a sample of 794 older adults in Sweden . The questionnaire aims to assess changes in cognitive function over the preceding 1 year. The items are rated on a 5-point Likert scale ranging from very seldom = 1, seldom =2, sometimes = 3, often = 4, and very often = 5. EFA was performed and the scale showed sufficient structural validity (the Kaiser-Meyer-Olkin measure of sampling adequacy = 0.95, Bartlett’s test of sphericity: v2 = 20,254, p < 0.001). Moreover, internal consistency was considered sufficient with a reported Cronbach’s α of 0.90. Convergent validity was evaluated against another measure of self-report cognitive decline, the PRMQ , which was of sufficient quality (r = 0.40, p < 0.001). The same questionnaire was evaluated by the same first author to assess the structural validity of a refined version that has 20 more items using CFA. The reported results indicate sufficient structural validity of the refined version of the CDQ (S-B χ2 = 558.5, df = 165, p < 0.000, RMSEA = 0.046 (CI; 0.042–0.050), SRMR = 0.057, CFI = 0.98).
The Subjective Memory Complaints Questionnaire (SMCQ) is a 14-item scale that assesses subjective memory decline . The authors report developing and validating the questionnaire in a sample of 1651 older adults in South Korea (mean age 74.3, SD = 8.2). The response options are either yes or no. The highest possible total score on the SMCQ is 14 points (SMCQ-T): 4 points for the judgment of global memory (SMCQ-G) and 10 points for everyday memory (SMCQ-E). The authors demonstrated that the SMCQ has sufficient structural validity by performing CFA. The reported goodness-of-fit index (GFI), comparative fit index (CFI), Tucker-Lewis index (TLI), and root mean square error of approximation (RMSEA) indices for model-fitting were 0.961, 0.929, 0.940, and 0.54, respectively. The internal consistency of the SMCQ was good and was considered to be sufficient (Cronbach’s α: SMCQ-T = 0.864, SMCQ-G = 0.827, SMCQ-E = 0.694). The authors also report sufficient test-retest reliability of the SMCQ, SMCQ-G, and SMCQ-E (0.828, p = 0.001; 0.471, p = 0.03; and 0.836, p = 0.001, respectively).
We conducted a systematic review to evaluate the reported measurement properties of SCD PROMs that are used to assess and detect SCD in older adults in the context of AD. We identified and included 16 studies that developed and/or validated 17 self-report measures that assess SCD. We examined the reported development procedure and the internal structure of the measures and provided a comprehensive evaluation of their risk of bias and methodological quality. Our findings suggest that currently available SCD PROMs do not address important psychometric properties.
SCD is an emerging construct that is being considered as one of the earliest clinical symptoms of AD [7, 12]. Our findings suggest that several SCD PROMs show sufficient structural validity and internal consistency. However, despite this, we were not able to formulate a recommendation for which SCD measure is most adequate to use. This is due to several factors. First, none of the included studies conducted a content validity study. Content validity is the extent to which a measure truly captures the construct that it was developed to measure . It is considered the most important, yet most challenging aspect of a PROM development and validation process . It involves a thorough procedure of asking patients and professionals to ensure that all items included in a measure are relevant to the construct under evaluation, are well comprehended by both the target population and professionals (e.g., health professionals or researchers), and are comprehensive enough to include all important aspects of the construct [21, 22]. Because no content validity evaluation was available, and as recommended by the COSMIN guidelines, we could not evaluate the overall quality of the PROM nor provide an assessment of the confidence of collected evidence to present a recommended SCD PROM. Secondly, only one study reported the administration time of the developed PROM (the spatial navigation questionnaire, average time of administration = 5 min) . This questionnaire, however, evaluates only subjective change in only one cognition domain. The majority of the included studies also failed to specify the mode (e.g., pen-and-paper versus electronic) and the proposed setting of administration (e.g., clinical setting versus at home). Without knowing how long each questionnaire takes or how it should be administered, it is difficult to infer the level of feasibility of the questionnaire at hand.
Some limitations of the present systematic review are worth mentioning. For inclusion in the systematic review, studies needed to have reported psychometric evaluation of questionnaires assessing SCD in older adults in the context of AD. This led to the exclusion of self-reported to questionnaires that were developed to assess the construct of SCD in general and were developed or validated in other populations. Indeed, valuable insights can be gathered from these questionnaires, some of which may be well-validated questionnaire that reliably assesses the same construct of interest . However, including indirect evidence may introduce heterogeneity because participants in different studies would differ greatly in their sociodemographic and risk factors than those with or at risk of AD . Moreover, our aim was to understand the current status of SCD self-reported questionnaire that are specifically designed to assess SCD related to AD. Therefore, the review team decided to only consider evidence that addressed and included populations with or at risk of AD and that validated questionnaires for the sole purpose of identifying SCD in the context of AD. In addition, all identified 17 PROMs were developed and validated in high-income settings (i.e., European countries the USA, South Korea). This sheds light on the importance of further cross-cultural validation of SCD measures, not only in other languages but also more importantly, in low-and-middle-income settings as well.
In conclusion, the results of this systematic review highlight the need for questionnaires that address important measurement properties in order to reliably assess SCD in older adults. A well-validated measure that is also feasible can aid in identifying older adults at risk of AD. Early identification would not only assist in including the right population for clinical trials, but also allow people the opportunity to be followed up closely and be offered meaningful intervention as early as possible.
Availability of data and materials
Data is available upon reasonable request.
Price JL, Davis PB, Morris JC, White DL. The distribution of tangles, plaques and related immunohistochemical markers in healthy aging and Alzheimer’s disease. Neurobiol Aging. 1991;12:295–312 Available from: https://linkinghub.elsevier.com/retrieve/pii/0197458091900066 [cited 18 Oct 2021].
Yang Y, Mufson EJ, Herrup K. Neuronal cell death is preceded by cell cycle events at all stages of Alzheimer’s disease. J Neurosci. 2003;23:2557–63 Available from: https://www.jneurosci.org/lookup/doi/10.1523/JNEUROSCI.23-07-02557.2003 [cited 18 Oct 2021].
Jack CR, Albert MS, Knopman DS, McKhann GM, Sperling RA, Carrillo MC, et al. Introduction to the recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:257–62.
McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR, Kawas CH, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:263–9.
Sperling RA, Aisen PS, Beckett LA, Bennett DA, Craft S, Fagan AM, et al. Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:280–92.
Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, et al. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:270–9.
Jessen F, Amariglio RE, van Boxtel M, Breteler M, Ceccaldi M, Chételat G, et al. A conceptual framework for research on subjective cognitive decline in preclinical Alzheimer’s disease. Alzheimers Dement. 2014;10:844–52.
Jessen F, Spottke A, Boecker H, Brosseron F, Buerger K, Catak C, et al. Design and first baseline data of the DZNE multicenter observational study on predementia Alzheimer’s disease (DELCODE). Alzheimers Res Ther. 2018;10:15.
Jessen F, Wiese B, Bachmann C, Eifflaender-Gorfer S, Haller F, Kölsch H, et al. Prediction of dementia by subjective memory impairment: effects of severity and temporal association with cognitive impairment. Arch Gen Psychiatry. 2010;67:414–22.
van Oijen M, de Jong FJ, Hofman A, Koudstaal PJ, Breteler MMB. Subjective memory complaints, education, and risk of Alzheimer’s disease. Alzheimers Dement. 2007;3:92–7.
Reisberg B, Shulman MB, Torossian C, Leng L, Zhu W. Outcome over seven years of healthy adults with and without subjective cognitive impairment. Alzheimers Dement. 2010;6:11–24.
Tales A, Jessen F, Butler C, Wilcock G, Phillips J, Bayer T. Subjective cognitive decline. J Alzheimers Dis. 48:S1–3 Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4923739/. [cited 18 Oct 2021].
Xu X, Li W, Tao M, Xie Z, Gao X, Yue L, et al. Effective and accurate diagnosis of subjective cognitive decline based on functional connection and graph theory view. Front Neurosci. 2020;14:1016 Available from: https://www.frontiersin.org/article/10.3389/fnins.2020.577887 [cited 18 Oct 2021].
Rabin LA, Smart CM, Crane PK, Amariglio RE, Berman LM, Boada M, et al. Subjective cognitive decline in older adults: an overview of self-report measures used across 19 international research studies. J Alzheimers Dis. 2015;48:S63–86.
Bassett SS, Folstein MF. Memory complaint, memory performance, and psychiatric diagnosis: a community study. J Geriatr Psychiatry Neurol. 1993;6:105–11.
Jessen F, Feyen L, Freymann K, Tepest R, Maier W, Heun R, et al. Volume reduction of the entorhinal cortex in subjective memory impairment. Neurobiol Aging. 2006;27:1751–6 Available from: https://www.sciencedirect.com/science/article/pii/S0197458005003416 [cited 18 Oct 2021].
Jungwirth S, Fischer P, Weissgram S, Kirchmeyr W, Bauer P, Tragl K-H. Subjective memory complaints and objective memory impairment in the Vienna-Transdanube Aging Community. J Am Geriatr Soc. 2004;52:263–8 Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1532-5415.2004.52066.x [cited 10 Dec 2019].
Dufouil C, Fuhrer R, Alpérovitch A. Subjective cognitive complaints and cognitive decline: consequence or predictor? The epidemiology of vascular aging study: cognitive complaints and cognitive decline. J Am Geriatr Soc. 53:616–21 Available from: https://onlinelibrary.wiley.com/doi/10.1111/j.1532-5415.2005.53209.x [cited 18 Oct 2021].
Treves TA, Verchovsky R, Klimovitzky S, Korczyn AD. Incidence of dementia in patients with subjective memory complaints. Int Psychogeriatr. 2005;17:265–73 Available from: https://www.cambridge.org/core/product/identifier/S1041610205001596/type/journal_article [cited 18 Oct 2021].
Prinsen CA, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27:1147–57.
Mokkink LB, de Vet HCW, Prinsen CA, Patrick DL, Alonso J, Bouter LM, et al. COSMIN Risk of Bias Checklist for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27:1171–9.
Terwee CB, Prinsen CA, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27:1159–70.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100 Available from: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1000100 [cited 18 Oct 2021]. Public Library of Science.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;n71 Available from: https://www.bmj.com/lookup/doi/10.1136/bmj.n71 [cited 18 Oct 2021].
Reisberg B, Ferris SH, de Leon MJ, Crook T. The Global Deterioration Scale for assessment of primary degenerative dementia. Am J Psychiatry. 1982;139:1136–9.
Hong YJ, Lee J-H. Subjective cognitive decline and Alzheimer’s disease spectrum disorder. Dement Neurocogn Disord. 2017;16:40–7.
Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5:210 Available from: http://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-016-0384-4 [cited 18 Oct 2021].
Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–82 Available from: http://content.apa.org/journals/bul/76/5/378 [cited 18 Oct 2021].
RStudio Team. RStudio: integrated development for R. RStudio [Internet]. Boston: RStudio, PBC; 2020. Available from: http://www.rstudio.com/
Allison SL, Rodebaugh TL, Johnston C, Fagan AM, Morris JC, Head D. Developing a spatial navigation screening tool sensitive to the preclinical Alzheimer disease continuum. Arch Clin Neuropsychol. 2019;34:1138–55 Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85074745155&doi=10.1093%2farclin%2facz019&partnerID=40&md5=d4d4ac1b05206a64113351648ad110ca.
Avila-Villanueva M, Rebollo-Vazquez A, Ruiz-Sanchez de Leon JM, Valenti M, Medina M, Fernandez-Blazquez MA. Clinical relevance of specific cognitive complaints in determining mild cognitive impairment from cognitively normal states in a study of healthy elderly controls. Front Aging Neurosci. 2016;8:233.
Chipi E, Frattini G, Eusebi P, Mollica A, D’Andrea K, Russo M, et al. The Italian version of cognitive function instrument (CFI): reliability and validity in a cohort of healthy elderly. Neurol Sci. 2018;39:111–8 Available from: http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2017-48143-001&site=ehost-live.
Crook TH, Feher EP, Larrabee GJ. Assessment of memory complaint in age-associated memory impairment: the MAC-Q. Int Psychogeriatr. 1992;4:165–76 Available from: https://www.cambridge.org/core/product/identifier/S1041610292000991/type/journal_article [cited 12 Mar 2021].
Crowe M, Andel R, Wadley V, Cook S, Unverzagt F, Marsiske M, et al. Subjective cognitive function and decline among older adults with psychometrically defined amnestic MCI. Int J Geriatr Psychiatry. 2006;21:1187–92 Available from: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L46029223.
Gifford KA, Liu D, Romano RR, Jones RN, Jefferson AL. Development of a subjective cognitive decline questionnaire using item response theory: a pilot study. Alzheimers Dement (Amst). 2015;1:429–39 Available from: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L607032618.
Gilewski MJ, Zelinski EM, Schaie KW. The Memory Functioning Questionnaire for assessment of memory complaints in adulthood and old age. Psychol Aging. 1990;5:482–90.
La Joie R, Perrotin A, Egret S, Pasquier F, Tomadesso C, Mézenge F, et al. Qualitative and quantitative assessment of self-reported cognitive difficulties in nondemented elders: association with medical help seeking, cognitive deficits, and β-amyloid imaging. Alzheimers Dement (Amst). 2016;5:23–34 Available from: https://onlinelibrary.wiley.com/doi/abs/10.1016/j.dadm.2016.12.005 [cited 15 Oct 2021].
Lubitz AF, Eid M, Niedeggen M. Complainer profile identification (CPI): properties of a new questionnaire on subjective cognitive complaints. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2018;25:99–121 Available from: http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2018-04494-006&site=ehost-live.
Papaliagkas V, Papantoniou G, Tsolaki M, Moraitou D. Self-report instruments of cognitive failures as screening tools for subjective cognitive impairment in older adults. Hell J Nucl Med. 2017;20:58–70 Available from: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L627820604.
Rami L, Mollica MA, García-Sanchez C, Saldaña J, Sanchez B, Sala I, et al. The Subjective Cognitive Decline Questionnaire (SCD-Q): a validation study. J Alzheimers Dis. 2014;41:453–66 Available from: http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2014-27636-012&site=ehost-live.
Rattanabannakit C, Risacher SL, Gao S, Lane KA, Brown SA, McDonald BC, et al. The cognitive change index as a measure of self and informant perception of cognitive decline: relation to neuropsychological tests. J Alzheimers Dis. 2016;51:1145–55 Available from: http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2016-18788-019&site=ehost-live.
Valech N, Tort-Merino A, Coll-Padros N, Olives J, Leon M, Rami L, et al. Executive and language subjective cognitive decline complaints discriminate preclinical Alzheimer’s disease from normal aging. J Alzheimers Dis. 2018;61:689–703.
Vestergren P, Rönnlund M, Nyberg L, Nilsson L-G. Development of the Cognitive Dysfunction Questionnaire (CDQ) in a population based sample: the Cognitive Dysfunction Questionnaire. Scand J Psychol. 2011;52:218–28 Available from: https://onlinelibrary.wiley.com/doi/10.1111/j.1467-9450.2010.00861.x [cited 15 Oct 2021].
Vestergren P, Rönnlund M, Nyberg L, Nilsson L-G. Multigroup confirmatory factor analysis of the cognitive dysfunction questionnaire: instrument refinement and measurement invariance across age and sex: the Cognitive Dysfunction Questionnaire. Scand J Psychol. 2012;53:390–400 Available from: https://onlinelibrary.wiley.com/doi/10.1111/j.1467-9450.2012.00970.x [cited 15 Oct 2021].
Youn JC, Kim KW, Lee DY, Jhoo JH, Lee SB, Park JH, et al. Development of the subjective memory complaints questionnaire. Dement Geriatr Cogn Disord. 2009;27:310–7 Available from: http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L354497162.
Crook TH, Larrabee GJ. A self-rating scale for evaluating memory in everyday life. Psychol Aging. 1990;5:48–57 Available from: http://doi.apa.org/getdoi.cfm?doi=10.1037/0882-7922.214.171.124 [cited 16 Oct 2021].
Lachman ME, Baltes P, Nesselroade JR, Willis SL. Examination of personality-ability relationships in the elderly: the role of the contextual (interface) assessment mode. J Res Pers. 1982;16:485–501 Available from: https://linkinghub.elsevier.com/retrieve/pii/0092656682900071 [cited 16 Oct 2021].
McNair: Self-assessment of cognitive deficits - Google Scholar. Available from: https://scholar.google.com/scholar_lookup?title=Self-assessment%20of%20cognitive%20deficits.%20Assessment%20in%20geriatric%20psychopharmacology&author=D.M.%20McNair&publication_year=1983. [cited 16 Oct 2021]
Broadbent DE, Cooper PF, FitzGerald P, Parkes KR. The Cognitive Failures Questionnaire (CFQ) and its correlates. Br J Clin Psychol. 1982;21:1–16 Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.2044-8260.1982.tb01421.x [cited 16 Oct 2021].
Smith G, Della Sala S, Logie RH, Maylor EA. Prospective and retrospective memory in normal ageing and dementia: a questionnaire study. Memory. 2000;8:311–21.
Mulrow C. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med. 1997;127:989 Available from: http://annals.org/article.aspx?doi=10.7326/0003-4819-127-11-199712010-00008 [cited 18 Oct 2021].
The authors would like to thank Dr. Marta Fadda for providing feedback on the final version of the manuscript.
The systematic review was not funded.
Ethics approval and consent to participate
The systematic review did not require an ethical approval from the local ethics board.
Consent for publication
All co-authors revised and provided their consent to publish the final version of the manuscript.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Search strategy. Table S1. Reported results of the psychometric properties of the included SCD PROMs. Table S2. Summary of the assessment rating for the quality of measurement properties of the included SD PROMs. Table S3. Overview of excluded studies after full text revision, with citation.
About this article
Cite this article
Ibnidris, A., Robinson, J.N., Stubbs, M. et al. Evaluating measurement properties of subjective cognitive decline self-reported outcome measures: a systematic review. Syst Rev 11, 144 (2022). https://doi.org/10.1186/s13643-022-02018-y
- Cognitive dysfunction
- Preclinical AD
- Measurement properties