Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Measurement properties of asthma-specific quality-of-life measures: protocol for a systematic review

  • Christian Apfelbacher1, 2Email author,
  • Priya Paudyal2,
  • Alpaslan Bülbül2 and
  • Helen Smith2
Systematic Reviews20143:83

https://doi.org/10.1186/2046-4053-3-83

Received: 27 March 2014

Accepted: 16 July 2014

Published: 24 July 2014

Abstract

Background

Asthma is a frequent chronic inflammatory disease of the airways, and the assessment of health-related quality of life (HrQoL) is important in both research and routine care. Various asthma-specific measures of HrQoL exist but there is uncertainty which measures are best suited for use in research and routine care. Therefore, the aim of the proposed research is a comprehensive systematic assessment of the measurement properties of the existing measures that were developed to measure asthma-specific quality of life.

Methods/design

This study is a systematic review of the measurement properties of asthma-specific measures of health-related quality of life. PubMed and Embase will be searched using a selection of relevant search terms. Eligible studies will be primary empirical studies evaluating, describing or comparing measurement properties of asthma-specific HRQL tools. Eligibility assessment and data abstraction will be performed independently by two reviewers. Evidence tables will be generated for study characteristics, instrument characteristics, measurement properties and interpretability. The quality of the measurement properties will be assessed using predefined criteria. Methodological quality of studies will be assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. A best evidence synthesis will be undertaken if more than one study have investigated a particular measurement property.

Discussion

The proposed systematic review will produce a comprehensive assessment of measurement properties of existing measures of asthma-specific health-related quality of life. We also aim to derive recommendations in order to help researchers and practitioners alike in the choice of instrument.

Trial registration

PROSPERO registration number: CRD42014010491.

Keywords

AsthmaHealth-related quality of lifeValidityReliabilityResponsiveness to change

Background

Asthma is a chronic inflammatory disease with variable airway obstruction, characterized by episodes of coughing, wheezing, breathlessness and chest tightness. It is estimated that around 300 million people worldwide suffer from asthma and that globally asthma accounts for about one in every 250 deaths [1]. By 2025, there may be 100 million more people with asthma. The number of disability-adjusted life years (DALYs) lost to asthma is comparable to the DALYs lost for diabetes, cirrhosis of the liver or schizophrenia.

Various medical and non-medical interventions exist to alleviate asthma, many of which have been assessed in randomized controlled trials. Because different outcome measures of various domains (such as disease severity, quality of life, symptoms) are used in different trials, the findings from these trials are difficult to compare. The lack of standardized asthma outcome measurement currently makes truly evidence-based decision making difficult, if not impossible.

One frequently used outcome is ‘health-related quality of life’ (HRQoL) which is a patient-reported outcome (PRO). According to the definition of the U.S. Food and Drug Administration (FDA), a PRO is defined as ‘any report of the status of a patient's health condition that comes directly from the patient, without interpretation of the patient's responses by a physician or anyone else’ [2]. Several reviews of PRO instruments for asthma have been conducted.

A published structured review identified six common asthma-specific quality-of-life measures: the Juniper Asthma Quality of Life Questionnaire (AQLQ-J) [3], the Sydney Asthma Quality of Life Questionnaire (AQLQ-S) [4], the Living with Asthma Questionnaire (LWAQ) [5], the St George's Respiratory Questionnaire (SGRQ) [6, 7], the Quality of Life for Respiratory Illness Questionnaire (QOL-RIQ) [8] and the Rhinasthma questionnaire [9]. The measures were reviewed using seven different criteria: conceptual and measurement model, reliability, validity, interpretability, burden, administration format and translations. The review concluded that the instruments differed in almost all criteria, and therefore, it cannot be assumed that they measure the same thing. The authors recommend to select those questionnaires that were designed for asthma and that do not assess symptoms as part of quality of life (QoL). These requirements are fulfilled by the Sydney Asthma QoL Questionnaire (AQLQ-S) [4] and the Living with Asthma Questionnaire (LWAQ) [5]. However, it is also stated that it remains unclear which of the questionnaires best reflects patient perception of QoL.

The Patient-reported Outcome Measurement Group (POMG) in England completed a comprehensive structured review of patient-reported outcome measures for people with asthma and provided recommendations to the Department of Health [10]. The difference to the structured review is that it not only encompasses asthma-specific quality-of-life measures, but also measures of health status, asthma control, utilities and symptoms. Twenty-two asthma-specific measures were evaluated. Based on the volume of evaluations and good measurement and operational characteristics, the original Juniper Asthma Quality of Life Questionnaire [3], the standardized Juniper Asthma Quality of Life Questionnaire [11] and the mini Juniper Asthma Quality of Life Questionnaire [12] as well as the Sydney Asthma Quality of Life Questionnaire [4] were presented to a multidisciplinary panel for discussion. Based on appraisal of evidence by the POMG and taking account of ratings and comments from the panel, the mini AQLQ was recommended as an asthma-specific instrument. Its ease of use and patient acceptability as well as the good concordance between postal and supervised administration was considered to be an important characteristic for measuring outcomes in NHS clinical care.

The third important review comes from the USA. In the light of a lack of adequate outcomes standardization, several National Institutes of Health (NIH) organizations that support asthma research as well as the Agency for Healthcare Research and Quality in the US agreed to a drive towards outcome standardization. As part of this effort, the published documentation relating to asthma-specific quality-of-life measures was reviewed [13]. In the review, the existing instruments were classified as follows:

  • Core outcomes: selective set of asthma outcomes to be considered by participating NIH institutes and other federal agencies as requirements for institute-/agency-initiated funding of clinical trials and large observational studies in asthma

  • Supplemental outcomes: asthma outcomes for which standard definitions can or have been developed, methods for measurement can be specified, and validity has been proved but whose inclusion in funded clinical asthma research will be optional

  • Emerging outcomes: asthma outcomes that have the potential to (1) expand and/or improve current aspects of disease monitoring and (2) improve translation of basic and animal model-based asthma research into clinical research. Emerging outcomes may be new or may have been previously used in asthma clinical research, but they are not yet standardized and require further development and validation

The US review identified 9 instruments for adults, the ABP [14], the Asthma Impact Survey [15], the AQLQ-J-s [11], the mini AQLQ-J [12], the LWAQ [16], the modified AQLQ-S [17], the Asthma Short Form [18], the SGRQ [6, 7] and the AQ-20 [17].

None of these qualified as core outcome because they predominantly measured indicators of asthma control (symptoms and/or functional status), failed to provide a distinct, reliable score measuring all key dimensions of the intended construct and/or lacked adequate psychometric data.

All three reviews have failed to perform a systematic literature search and have not included a systematic assessment of the methodological quality of the included studies. The proposed review will therefore systematically assess the measurement properties of asthma-specific quality-of-life measures and include an assessment of the methodological quality of all included studies.

Methods/design

This study is a systematic review of the measurement properties of asthma-specific measures of health-related quality of life.

Eligible measures

All measures which are specifically designed to measure health-related quality of life in asthma are eligible.

Eligible studies

Inclusion criteria

A study will be included if it is a full text paper, published in English and describes the development (‘inauguration paper’) and/or evaluation of the measurement properties (‘validation paper’) of an asthma-specific measure of HRQoL. The study population should be adults (≥18 years) with asthma. The study should be published as a full text paper and the instrument should be a self-report questionnaire.

Exclusion criteria: Articles reporting on interview instruments or instruments for proxies will not be considered for the purpose of this review. Articles that report an eligible measure, e.g. as an outcome in a clinical trial without any explicit validation will not be considered eligible.

Literature search

A systematic literature search will be performed in PubMed and EMBASE. Blocks of search terms will be used relating to the following aspects:

  •  Construct of interest (asthma-specific health-related quality of life): here, the broad search term ‘quality of life’ will be used.

  •  Target population (adult asthma patients): here, different combinations will be used like ‘allergic’ or ‘atopic’ or ‘bronchial’ and each will be combined with ‘asthma’. The search will be limited to humans.

  •  Measurement properties: The highly sensitive PubMed search filter for finding studies on measurement properties developed by Terwee et al. will be used to identify relevant articles [19]. This filter has a sensitivity of 97.4% and a precision of 4.4%. A selection of relevant search terms will be used in EMBASE, which has been used before in reviews.

For each of these search strategies, a thorough list of synonyms will be collated using index terms (e.g. MESH terms in PubMed) linked with other free text words. The synonyms will be combined with the conjunction ‘OR’. After that, the searches designed according to the three main aspects will be combined with ‘AND’ in order to get to the list of publications from which the relevant ones will be picked. A further search will be conducted with the names of instruments found in the original search. These names should be combined with AND with the requirements for the target population and measurement properties. The references of all the included relevant articles will also be screened. In addition, websites of relevant professional organizations and institutions will be searched. For all searches, search dates will be provided in the review.

Study selection

In a first phase, titles and abstracts will be assessed for eligibility. Full text articles will be obtained for the remaining abstracts and again be assessed for eligibility. Each citation will be judged for eligibility independently by two reviewers. Disagreements will be resolved by discussion of all reviewers.

Data extraction

Two reviewers will independently extract data from each article included. Relevant data from all included articles will be summarized in evidence tables. Evidence tables will contain the following information: characteristics of the instrument (name of measure, domains measured, number of items), characteristics of the study population (geographical location, gender, age, co-morbidities), results for conceptual models and measurement properties (reliability, validity, responsiveness to change according to Table 1) and results for interpretability (including minimal important difference). The evidence tables will be pilot-tested.
Table 1

Quality criteria for measurement properties

Property

Rating

Quality criteria

Reliability

  

  Internal consistency

+

(Sub)scale unidimensional AND Cronbach's alpha(s) ≥0.70

?

Dimensionality not known OR Cronbach's alpha not determined

-

(Sub)scale not unidimensional OR Cronbach's alpha(s) <0.70

  Measurement error

+

MIC > SDC OR MIC outside the LOA

?

MIC not defined

-

MIC ≤ SDC OR MIC equals or inside LOA

  Reliability

+

ICC/weighted kappa ≥0.70 OR Pearson's r ≥ 0.80

?

Neither ICC/weighted kappa, nor Pearson's r determined

-

ICC/weighted kappa <0.70 OR Pearson's r < 0.80

Validity

  

  Content validity

+

All items are considered to be relevant for the construct to be measured, for the target population and for the purpose of the measurement AND the questionnaire is considered to be comprehensive

?

Not enough information available

-

Not all items are considered to be relevant for the construct to be measured, for the target population and for the purpose of the measurement OR the questionnaire is considered not to be comprehensive

Construct validity

  

  Structural validity

+

Factors should explain at least 50% of the variance

?

Explained variance not mentioned

-

Factors explain <50% of the variance

  Hypothesis testing

+

Correlation with an instrument measuring the same construct ≥0.50 OR at least 75% of the results are in accordance with the hypotheses AND correlation with related constructs is higher than with unrelated constructs

?

Solely correlations determined with unrelated constructs

-

Correlation with an instrument measuring the same construct <0.50 OR <75% of the results are in accordance with the hypotheses OR correlations with related constructs are lower than with unrelated constructs

Responsiveness

  

  Responsiveness

+

Correlation with changes on instruments measuring the same construct ≥0.50 OR at least 75% of the results are in accordance with the hypotheses OR AUC ≥0.70 AND correlation with changes in related constructs are higher than with unrelated constructs

?

Solely correlations determined with unrelated constructs

-

Correlations with changes on instruments measuring the same construct <0.50 OR <75% of the results are in accordance with the hypotheses OR AUC <0.70 OR correlations with changes in related constructs are lower than with unrelated constructs

Plus sign indicates positive rating; question mark, indeterminate rating and minus sign, negative rating. MIC minimal important change, SDC smallest detectable change, LOA limits of agreement, ICC intraclass correlation coefficient, AUC area under the curve.

Assessment of the measurement properties of instruments

The predefined quality criteria for rating the measurement properties of instruments recommended by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) group will be used to assess the measurement properties of the measures [20] (Table 1). These relate to the following measurement properties and aspects of measurement properties: reliability (internal consistency, measurement error, reliability), validity (content validity, structural validity, hypothesis testing) and responsiveness. In addition, we will consider whether the development of any instrument included in the systematic review was based on an a priori conceptual framework/model.

Assessment of the methodological quality of included studies

The COSMIN checklist [2123] will be used to evaluate the methodological quality of included studies. In the COSMIN checklist (http://www.cosmin.nl), four domains are distinguished (reliability, validity, responsiveness and interpretability) with related measurement properties and aspects of measurement properties. For each of the measurement properties, the COSMIN checklist consists of 5–18 items covering methodological standards (organized in nine boxes for the nine measurement properties). In addition, each item can be scored on a four-point scale (i.e. ‘poor’ , ‘fair’ , ‘good’ , ‘excellent’). Taking the lowest rating for each item in one box, an overall quality score (poor, fair, good, excellent) is obtained for each measurement property separately.

Best evidence synthesis

If several studies exist for one measure, findings will be synthesized by combining them, based on number and methodological quality of the studies and consistency of results as previously suggested [24]. The criteria for synthesizing evidence are summarized in Table 2.
Table 2

Levels of evidence for the overall quality of a measurement property

Level

Rating

Criteria

Strong

+++ or ---

Consistent findings in multiple studies of good methodological quality OR in one study of excellent methodological quality

Moderate

++ or --

Consistent findings in multiple studies of fair methodological quality OR in one study of good methodological quality

Limited

+ or -

One study of fair methodological quality

Conflicting

+/-

Conflicting findings

Unknown

?

Only studies of poor methological quality

Plus sign indicates positive rating; question mark, indeterminate rating and minus sign, negative rating.

Discussion

The proposed systematic review will produce a comprehensive assessment of measurement properties of existing measures of asthma-specific health-related quality of life. We will highlight the major findings of the review and describe evidence in terms of grade supporting or not supporting the use of any given asthma-specific HRQL tool. We will highlight problems and limitations that we will find across the reviewed tools. The strengths and limitations in the identified evidence (e.g. relating to amount of evidence, validity, feasibility) will be presented and discussed. Finally, we will report about strengths and limitations of our review and highlight future research and policy implications.

We also aim to derive recommendations in order to help researchers and practitioners alike in the choice of instrument. For each instrument identified in the review, a standardized recommendation for usage or required future validation work will be made depending on the best evidence synthesis.

Four categories of recommendation will be made as follows:
  1. a)

    Outcome measure achieves positive ratings (at least ‘+’) for all measurement properties and is recommended for use.

     
  2. b)

    Outcome measure achieves positive ratings (at least ‘+’) for at least two measurement properties, but performance in all other required measurement properties is unclear, so that the outcome measure has the potential to be recommended in the future depending on the results of further validation studies.

     
  3. c)

    Outcome measure has low quality in at least one measurement property (at least one ‘-’ rating) and therefore is not recommended to be used any more.

     
  4. d)

    Outcome measure has (almost) not been validated. Its performance in all or most relevant measurement properties is unclear, so that it is not recommended to be used until further validation studies clarify its quality.

     

Abbreviations

HrQoL

Health-related quality of life

COSMIN

COnsensus-based Standards for the selection of health Measurement INstruments.

Declarations

Authors’ Affiliations

(1)
Medical Sociology, Institute of Epidemiology and Preventive Medicine, University of Regensburg
(2)
Division of Public Health and Primary Care, Brighton and Sussex Medical School

References

  1. Masoli M, Fabian D, Holt S, Beasley R: The global burden of asthma: executive summary of the GINA dissemination committee report. Allergy. 2004, 59: 469-478.View ArticlePubMedGoogle Scholar
  2. U.S. Department of Health and Human Services Food and Drug Administration: Food and Drug Administration guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. 2009. Book U.S. Department of Health and Human Services Food and Drug Administration: Food and Drug Administration guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. 2009, Available from: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm071324.pdf Google Scholar
  3. Juniper EF, Guyatt GH, Epstein RS, Ferrie PJ, Jaeschke R, Hiller TK: Evaluation of impairment of health related quality of life in asthma: development of a questionnaire for use in clinical trials. Thorax. 1992, 47: 76-83.View ArticlePubMedPubMed CentralGoogle Scholar
  4. Marks GB, Dunn SM, Woolcock AJ: A scale for the measurement of quality of life in adults with asthma. J Clin Epidemiol. 1992, 45: 461-472.View ArticlePubMedGoogle Scholar
  5. Hyland ME, Finnis S, Irvine SH: A scale for assessing quality of life in adult asthma sufferers. J Psychosom Res. 1991, 35: 99-110.View ArticlePubMedGoogle Scholar
  6. Jones PW, Quirk FH, Baveystock CM: The St George's respiratory questionnaire. Respir Med. 1991, 85 Suppl B: 25-31. discussion 33-27View ArticlePubMedGoogle Scholar
  7. Jones PW, Quirk FH, Baveystock CM, Littlejohns P: A self-complete measure of health status for chronic airflow limitation. The St George's respiratory questionnaire. Am Rev Respir Dis. 1992, 145: 1321-1327.View ArticlePubMedGoogle Scholar
  8. Maille AR, Koning CJ, Zwinderman AH, Willems LN, Dijkman JH, Kaptein AA: The development of the ‘Quality-of-life for respiratory illness questionnaire (QOL-RIQ)’: a disease-specific quality-of-life questionnaire for patients with mild to moderate chronic non-specific lung disease. Respir Med. 1997, 91: 297-309.View ArticlePubMedGoogle Scholar
  9. Baiardini I, Pasquali M, Giardini A, Specchia C, Passalacqua G, Venturi S, Braido F, Bonini S, Majani G, Canonica GW: Rhinasthma: a new specific QoL questionnaire for patients with rhinitis and asthma. Allergy. 2003, 58: 289-294.View ArticlePubMedGoogle Scholar
  10. Gibbons E, Fitzpatrick R: A structured review of patient-reported outcome measures for people with asthma: an update 2009. 2009, Available from: http://phi.uhce.ox.ac.uk/ Google Scholar
  11. Juniper EF, Buist AS, Cox FM, Ferrie PJ, King DR: Validation of a standardized version of the asthma quality of life questionnaire. Chest. 1999, 115: 1265-1270.View ArticlePubMedGoogle Scholar
  12. Juniper EF, Guyatt GH, Cox FM, Ferrie PJ, King DR: Development and validation of the mini asthma quality of life questionnaire. Eur Respir J. 1999, 14: 32-38.View ArticlePubMedGoogle Scholar
  13. Wilson SR, Rand CS, Cabana MD, Foggs MB, Halterman JS, Olson L, Vollmer WM, Wright RJ, Taggart V: Asthma outcomes: quality of life. J Allergy Clin Immunol. 2012, 129: S88-S123.View ArticlePubMedPubMed CentralGoogle Scholar
  14. Haave E, Hyland ME: Norwegian versions of the living with asthma questionnaire (LWAQ) and asthma bother profile (ABP), validation and comparison of two asthma groups. Scand J Psychol. 2004, 45: 163-167.View ArticlePubMedGoogle Scholar
  15. Schatz M, Mosen D, Kosinski M, Vollmer WM, O'Connor E, Cook EF, Zeiger RS: Validation of the asthma impact survey, a brief asthma-specific quality of life tool. Qual Life Res. 2007, 16: 345-355.View ArticlePubMedGoogle Scholar
  16. Hyland ME: The living with asthma questionnaire. Respir Med. 1991, 85 Suppl B: 13-16. discussion 33-17View ArticlePubMedGoogle Scholar
  17. Adams RJ, Ruffin RE, Smith BJ: Validity of a modified version of the marks asthma quality of life questionnaire. J Asthma. 2000, 37: 131-143.View ArticlePubMedGoogle Scholar
  18. Bayliss MS, Espindle DM, Buchner D, Blaiss MS, Ware JE: A new tool for monitoring asthma outcomes: the ITG asthma short form. Qual Life Res. 2000, 9: 451-466.View ArticlePubMedGoogle Scholar
  19. Terwee CB, Jansma EP, Riphagen II, de Vet HC: Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res. 2009, 18: 1115-1123.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, Bouter LM, de Vet HC: Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007, 60: 34-42.View ArticlePubMedGoogle Scholar
  21. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HC: The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010, 63: 737-745.View ArticlePubMedGoogle Scholar
  22. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, Bouter LM, de Vet HC: The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010, 10: 22-View ArticlePubMedPubMed CentralGoogle Scholar
  23. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HC: The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010, 19: 539-549.View ArticlePubMedPubMed CentralGoogle Scholar
  24. Furlan AD, Pennick V, Bombardier C, van Tulder M: 2009 updated method guidelines for systematic reviews in the Cochrane back review group. Spine. 2009, 34: 1929-1941.View ArticlePubMedGoogle Scholar

Copyright

© Apfelbacher et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Advertisement