Table of Contents
Chronic obstructive pulmonary disease (COPD) is characterized by persistent respiratory symptoms and airflow limitation and represents one of the leading causes of morbidity and mortality worldwide.1 Phenotyping of COPD patients by clinical features and biomarkers can inform decision-making and prognostication.2
Numerous biomarkers are routinely used to diagnose COPD, evaluate disease severity, and assess response to treatment. However, relatively few studies have focused on the variability of these biomarkers in clinically stable disease.3,4 Assessing repeatability and reproducibility allows for evaluation of the stability and reliability of measuring methods. Reliability depends on the population in which the measurements are made and not just technical errors with respect to the methodology and the magnitude of the measurement’s inherent variability. It is quantified using the intraclass correlation coefficient (ICC), which ranges between zero and one, with a value of one corresponding to zero measurement error and a value of zero meaning that all the variability in measurements is due to measurement error.5
In order to better understand the reliability of decision-making based on interpretation of biomarker values, we compared the variability of commonly assessed biomarkers over 3 months in clinically stable COPD.
We analyzed data from the COPD Phenotyping Study, a collaboration between the University of California Los Angeles (UCLA) and AMGEN (Thousand Oaks, CA).6 This prospective, observational cohort study was performed at UCLA from October 2015 to September 2018. The study was approved by the UCLA Institutional Review Board (IRB 14–000748), and written informed consent was obtained from all participants. The participants were 40 to 80 years of age with >10 pack-year smoking history and clinically stable COPD. For subjects with clinically stable COPD, clinical stability was defined as having been on stable medications for COPD for 3 months before Visit 1, not including systemic corticosteroids, and having no history of an exacerbation within the preceding 3 months. The clinically stable subjects were identified at the outpatient clinics of Ronald Reagan University of California and Los Angeles (UCLA) Medical Center, Santa Monica UCLA Medical Center. The study coordinator was present at these weekly clinics and identified potential subjects, administered consent, recorded vital signs, drew blood, collected nasal brushing and sputum samples, performed spirometry, and complete questionnaires and the functional exercise test. Individuals with obstructive lung disease other than COPD (eg, pure asthma) or with pulmonary parenchymal disease (eg, pulmonary fibrosis) were excluded.
Biomarkers were categorized into three groups: 1) lung function and physical performance; 2) respiratory symptoms and health-related quality of life (HRQoL); and 3) blood markers of inflammation. Lung function and physical performance biomarkers included FEV1, FVC, handgrip strength (HGS), and 6-minute walk distance (6MWD). Respiratory symptoms and HRQoL biomarkers included modified Medical Research Council (mMRC),7 COPD Assessment Test (CAT),8 Medical Outcomes Trust Short Form 12 (SF-12),9 Saint George’s Respiratory Questionnaire (SGRQ),10 and Veterans Specific Activity Questionnaire (VSAQ).11 Inflammatory biomarkers included C-reactive protein (CRP), erythrocyte sedimentation rate (ESR), fibrinogen, neutrophil–lymphocyte ratio (NLR), percentage blood eosinophils, blood eosinophil absolute count, and white blood cell count (WBC). All biomarkers were collected at baseline and 3-month follow-up. Spirometry was performed using a portable electronic spirometer (SpiroPro®, e Research Technology, Inc., Philadelphia, PA, USA) according to the 2005 American Thoracic Society (ATS)/European Respiratory Society (ERS) guidelines.12 Maximal HGS for each hand was averaged from three measurements obtained using a specialized dynamometer (Jamar; Asimow Engineering Co; Santa Monica, CA, USA). 6MWD was measured with the participant breathing room air, in accordance with the ATS 2002 guidelines.13 Peripheral venous blood samples were processed at UCLA’s central clinical laboratory.
Other Clinical Measurements
Medical history was obtained by questionnaire. Any history of cardiovascular disease, asthma, chronic bronchitis (defined as having chronic cough and sputum production for ≥3 months for 2 consecutive years), cancer, and exacerbation history was noted. Smoking history was reported in pack-years as equal to the number of cigarettes smoked per day × number of years smoked. Prior thoracic high-resolution computed tomography (CT) was available for a subset of participants, and quantitative image analysis was performed by thoracic radiologists for this study.
Categorical variables are expressed as numbers and percentages, and continuous variables are expressed as mean ± standard deviation or median and interquartile range. The paired t-test or Wilcoxon signed rank test was used for comparing variables at baseline and 3-month follow-up. The correlation between biomarkers at baseline and 3-month follow-up was assessed using Pearson’s correlation test or Spearman rank correlation test. Intraclass correlation coefficient (ICC) was calculated: ICC values <0.5, between 0.5 and 0.75, between 0.75 and 0.9, and >0.90 are indicative of poor, moderate, good, and excellent reliability, respectively.14 Participants with a large difference between the biomarker at baseline and 3-month follow-up were termed “outliers” and defined by >1.645 standard deviations (SD) above the mean difference (the top 5th percentile of a standard normal distribution) between the two time points. “Consistent outliers” were defined by variability both in participant-reported measures (symptom/health status) and in tests of lung function/functional status or inflammation. For assessing the association between outlier status and past or prospective exacerbations (3–12 months after enrollment), we included only consistent outliers who demonstrated worsening values of assessed biomarkers on their 3-month follow-up. SPSS version 25 (SPSS Inc., Chicago, Illinois, USA) was used for all statistical analysis, and a P value < 0.05 was considered statistically significant.
The characteristics of participants with COPD are shown in Table 1. The COPD Phenotyping Study enrolled 112 stable participants. Of these, 23 subjects were excluded because they did not complete spirometry (3 subjects), did not satisfy spirometric criteria (6 subjects), or did not complete follow-up visit (14 subjects). Finally, 89 subjects were included in the analysis (Figure 1). Their mean age was 70.5 years, and 54 (61%) were male. The participants had a median smoking history of 40 pack-years, and most of them had emphysema. Mean FEV1 was 62 ± 23% predicted. Of the 89 patients, 82 underwent CT scan, and 91.5% showed emphysema, 40.2% airway thickening, and 43.9% bronchiectasis. The most common comorbidity was cardiovascular disease. Frequently prescribed medications were short-acting beta-agonist (SABA), long-acting beta-agonist/inhaled corticosteroid (LABA/ICS), and long-acting muscarinic antagonist (LAMA). About a quarter of participants had a history of COPD exacerbations >3 months prior to enrollment, and no participants reported an exacerbation in between the two visits. Table 2 shows the comparison of the values of biomarkers within each of the three groups between baseline and the three-month follow-up visit. With the exception of SF-12 values, which showed a slight but significant mean increase after 3 months (68.2 vs 70.5, P = 0.01), there was no significant difference in the values of the other biomarkers between the two time points in the COPD participants over the follow-up period.
Table 1 Baseline Characteristics of Patients with Chronic Obstructive Pulmonary Disease
Table 2 Variables in Physiologic Parameters and Biomarkers at Baseline and 3-Month Follow-Up in Patients with Chronic Obstructive Pulmonary Disease
Figure 1 A flow chart for inclusion of patients with COPD.
Reliability of Measurements
The agreement between the values obtained at baseline and at the three-month follow-up visit (Figure 2, Supplement Table 1) was good for most variables but ranged from fair to excellent reliability. Of all the biomarkers, FEV1 was one of the most reliable with an excellent ICC (0.96). The CAT and SGRQ were also reliable with excellent ICCs (0.93 and 0.98, respectively). The 6MWD and ESR were less reliable with a moderate ICC (0.75, 0.75, and 0.48, respectively); only WBC had a poor ICC (0.48).
Outliers and Clinical Outcomes
Of the 89 participants in our cohort, 67 (75%) were outliers for at least one biomarker and 14 (16%) were consistent outliers, with variable results in more than one category of biomarker. Table 3 shows that there were fewer outliers in lung function and physical performance measurements and more outliers in patient-reported outcomes (respiratory symptoms and HRQoL). We identified a total of 14 (16%) “consistent outliers” in this cohort. Table 4 presents the comparison of clinical characteristics and outcomes between consistent outliers and others. Consistent outliers had significantly worse SF-12, SGRQ, and CAT, and more frequent chronic bronchitis. There was a nonsignificant trend toward lower FEV1 in consistent outliers.
Table 3 Frequency of Outliers in Each Biomarker in Patients with Chronic Obstructive Pulmonary Disease*
Table 4 Characteristics Between Consistent Outliers and Others in Patients with Chronic Obstructive Pulmonary Disease*
To further assess whether the outliers with worsening biomarker values on follow-up testing are at a higher risk of exacerbations, we examined the association between worsening biomarkers and baseline exacerbations over the 3–12 months prior to enrollment in the study, or during the follow-up period of one year after enrollment, however found no association with past or prospective exacerbations in this cohort (Supplement Table 2).
In this study, we evaluated the three-month variability of commonly used biomarkers in clinically stable COPD, and identified FEV1, CRP, and SGRQ as the most reliable COPD measurements over time. We also investigated the clinical significance of beyond-expected variability in biomarkers and found that, when present together, variability in multiple subjective (respiratory symptoms/HRQoL) and objective biomarkers (lung function/physical performance or inflammation) was associated with chronic bronchitis, worse baseline HRQoL and a trend toward worse baseline FEV1. Our data contribute to the understanding of biomarkers commonly used to assess the dynamics of COPD.
In contrast to studies evaluating repeatability and/or a shorter-term reproducibility,8,15–17 this study prospectively assessed changes in commonly used biomarkers over three months, an evaluation period frequently used in large observational cohorts,18 clinical trials,19–21 and clinical practice.22 This time period can be considered to be long enough to adequately reflect the natural course of the disease’s impact on the measured biomarkers, thus allowing for a real-life understanding of biomarker variability in clinically stable COPD. Based on our data, pulmonary function, patient-reported symptoms, health-related quality of life, and blood inflammatory markers remain relatively stable over a 3-month time period, with the majority of clinical biomarkers showing excellent or good agreement.
Spirometry is the most extensively studied COPD biomarker with regard to repeatability and reproducibility.23 FEV1 remains one of the most widely used biomarkers and is a strong predictor of morbidity and mortality in COPD.24,25 Consistent with these findings, our data show excellent agreement in FEV1 measured over the 3-month time interval, with better between-measurement agreement than seen for FVC, 6MWD or HGS. Our data show that 6MWD, a test routinely used as an outcome measure in clinical trials in COPD and a prognostic indicator either by itself or as part of a multidimensional index, demonstrated only moderate agreement (ICC = 0.75) between the 3-month interval measurements. These results suggest caution when using 6MWD as a clinical end-point due to this test’s inherent variability although we note that our results showed poorer agreement in comparison to previously reported reproducibility for this measure.26,27
Blood biomarkers of inflammation, including ESR, fibrinogen, CRP, white blood cell count, and eosinophil count, are associated with clinical outcomes such as exacerbations, hospitalizations, and mortality in COPD.28,29 In the ECLIPSE cohort, CRP levels demonstrated frequent within-in person variability, with only 21% of participants having a 3-month value that was within 25% of the baseline value.18 Other non-COPD focused studies have reported large intra-individual variability in CRP,30 and 46% of subjects changed CRP risk category (≥2mg/L for high risk and <2mg/L for low risk) at least once during one year follow-up.30 On the other hand, published data from the COPD literature suggests that CRP is a relatively stable marker with a coefficient of variance of 14.6% during an 18-month follow-up.31 Similarly, our data suggest CRP to be the least variable biomarker when compared to other commonly used biomarkers of inflammation.
Aside from physiologic and laboratory parameters, patient-reported outcomes (PRO) are increasingly recognized as an important measure of COPD disease severity and activity, as demonstrated by the US Food and Drug Administration endorsing patient-reported outcomes as a valid end-point in clinical trials.32 PRO measures are subjective and prone to variability: symptoms are reported to vary not only across seasons but also during a week or even a day.33,34 Nevertheless, the ICC (0.97) or coefficient of variation (19%) of SGRQ in patients with COPD or asthma, as previously reported, show that SGRQ is a repeatable measure.16,17 The CAT also has a good test–retest reliability and a good correlation (ICC = 0.80) with the COPD-specific version of SGRQ in patients with COPD.8 We demonstrate similarly excellent ICCs: 0.93 for CAT and 0.98 for SGRQ. These findings suggest that although CAT and SGRQ are subjective measures, they are reliable measures in clinically stable COPD.
To assess the overall significance of variability in repeated biomarker testing over a 3-month interval, we focused on variability that goes beyond expected in our COPD cohort. Variability beyond 95% of the normal distribution of values for multiple biomarkers including both PROs and biomarkers of lung function or inflammation in the same participant was associated with more advanced disease according to HRQoL measures, and a trend toward worse airflow obstruction, but no significant association with baseline or prospective exacerbations in this small cohort.
Several limitations require discussion. While all the testing was repeated in the same laboratory and with the same equipment, the observational nature of the study has less stringent control of real-life factors that could impact variability, such as adherence to COPD medications, timing of the administration of medications, or diurnal variability when measurements were made at different times of day. Since the study enrolled patients over almost three years, the impact of seasonal changes on biomarker variability was not accounted for. The study was not powered to demonstrate significant associations of biomarker instability with clinical outcomes such as exacerbations. The findings of our study may not be generalizable given the small number of participants and other biases related to the characteristics of this cohort from a large urban academic medical center.
Nevertheless, some strengths need to be emphasized. The results presented evaluate the stability of biomarkers in a well-profiled cohort and, given its observational nature, are less prone to biases of an interventional study. The follow-up period of three months is longer than that used by most of reproducibility studies,17,26,27,35–38 and allows for insight into the disease course over an interval of time that is often used in routine clinical care and in clinical trials. To our knowledge, this analysis is one of the few studies directly comparing the three-month stability of a relatively long list of commonly used biomarkers in COPD, analyzing the variability within different groups of biomarkers and evaluating the clinical significance of the variability of multiple biomarkers in individual patients with COPD.
In a prospective study of the longitudinal variability of commonly assessed biomarkers in clinically stable COPD, we showed fair to excellent agreement between baseline and three-month follow-up values, with FEV1, SGRQ, CAT and CRP being the most stable measures, while the six-minute walk test, leukocyte counts and ESR were more variable over the three-month period. Consistent variability in both subjective patient-reported outcomes and objective measures of lung function and inflammation was characteristic of more advanced COPD. Further studies are needed to better understand possible associations of instability of these biomarkers with clinical outcomes.
Data Sharing Statement
The data that support the findings of this study are available from the corresponding author, upon reasonable request.
Ethics Approval and Informed Consent
This study was approved by the UCLA Institutional Review Board (IRB# IRB 14-000748) and was performed in compliance with the Declaration of Helsinki. Written informed consent was obtained from all participants.
All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.
This study was supported by an investigator-initiated grant to Dr Christopher B Cooper from Amgen, Inc. (CTR#20147154).
JP is an employee of Amgen Inc. CBC reports personal fees from PulmonX, GlaxoSmithKline, NUVAIRA and MGC Diagnostics, outside the submitted work. DPT has consulted with AstraZeneca, Sunovion, Mylan and Theravance. IZB has consulted with Astra Zeneca, Grifols, Verona Pharma, Sanofi, Theravance, Viatris, Aerogen and Inhibrx and has received research grants from AMGEN, Aerogen, Theravance and Viatris and GE Healthcare. RGB reports personal fees from Viatris/Theravance Biopharma, outside the submitted work. The authors report no other conflicts of interest in this work.
1. Global Initiative for Chronic Obstructive Lung Disease. Global strategy for prevention, diagnosis, and management of COPD. Global Initiative for Chronic Obstructive Lung Disease; 2022. Available from: goldcopd.org/2022-gold-reports.
2. Mirza S, Benzo R. Chronic obstructive pulmonary disease phenotypes: implications for care. Mayo Clin Proc. 2017;92(7):1104–1112. doi:10.1016/j.mayocp.2017.03.020
3. Hollander Z, DeMarco ML, Sadatsafavi M, McManus BM, Ng RT, Sin DD. Biomarker development in COPD: moving from P values to products to impact patient care. Chest. 2017;151(2):455–467. doi:10.1016/j.chest.2016.09.012
4. Röpcke S, Holz O, Lauer G, et al. Repeatability of and relationship between potential COPD biomarkers in bronchoalveolar lavage, bronchial biopsies, serum, and induced sputum. PLoS One. 2012;7(10):e46207. doi:10.1371/journal.pone.0046207
5. Watson PF, Petrie A. Method agreement analysis: a review of correct methodology. Theriogenology. 2010;73(9):1167–1179. doi:10.1016/j.theriogenology.2010.01.003
6. Wilgus ML, Abtin F, Markovic D, et al. Panlobular emphysema is associated with COPD disease severity: a study of emphysema subtype by computed tomography. Respir Med. 2022;192:106717. doi:10.1016/j.rmed.2021.106717
7. Mahler DA, Ward J, Fierro-Carrion G, et al. Development of self-administered versions of modified baseline and transition dyspnea indexes in COPD. COPD. 2004;1(2):165–172. doi:10.1081/COPD-120030829
8. Jones PW, Harding G, Berry P, Wiklund I, Chen WH, Kline Leidy N. Development and first validation of the COPD assessment test. Eur Respir J. 2009;34(3):648–654. doi:10.1183/09031936.00102509
9. Ware J Jr, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220–233. doi:10.1097/00005650-199603000-00003
10. Meguro M, Barley EA, Spencer S, Jones PW. Development and validation of an improved, COPD-specific version of the St. George respiratory questionnaire. Chest. 2007;132(2):456–463. doi:10.1378/chest.06-0702
11. Myers J, Bader D, Madhavan R, Froelicher V. Validation of a specific activity questionnaire to estimate exercise tolerance in patients referred for exercise testing. Am Heart J. 2001;142(6):1041–1046. doi:10.1067/mhj.2001.118740
12. Miller MR, Hankinson J, Brusasco V, et al. Standardisation of spirometry. Eur Respir J. 2005;26(2):319–338. doi:10.1183/09031936.05.00034805
13. ATS Committee on Proficiency Standards for Clinical Pulmonary Function Laboratories. ATS statement: guidelines for the six-minute walk test. Am J Respir Crit Care Med. 2002;166(1):111–117. doi:10.1164/ajrccm.166.1.at1102
14. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–163. doi:10.1016/j.jcm.2016.02.012
15. Enright PL, Beck KC, Sherrill DL. Repeatability of spirometry in 18,000 adult patients. Am J Respir Crit Care Med. 2004;169(2):235–238. doi:10.1164/rccm.200204-347OC
16. Jones PW, Quirk FH, Baveystock CM, Littlejohns P. A self-complete measure of health status for chronic airflow limitation. The St. George’s respiratory questionnaire. Am Rev Respir Dis. 1992;145(6):1321–1327. doi:10.1164/ajrccm/145.6.1321
17. Wilson CB, Jones PW, O’leary CJ, Cole PJ, Wilson R. Validation of the St. George’s respiratory questionnaire in bronchiectasis. Am J Respir Crit Care Med. 1997;156(2 Pt 1):5365. doi:10.1164/ajrccm.156.2.9607083
18. Dickens JA, Miller BE, Edwards LD, Silverman EK, Lomas DA, Tal-Singer R. COPD association and repeatability of blood biomarkers in the ECLIPSE cohort. Respir Res. 2011;12(1):146. doi:10.1186/1465-9921-12-146
19. Ferguson GT, Feldman G, Pudi KK, et al. Improvements in lung function with nebulized revefenacin in the treatment of patients with moderate to very severe COPD: results from two replicate phase III clinical trials. COPD. 2019;6(2):154–165. doi:10.15326/jcopdf.6.2.2018.0152
20. Kerwin EM, D’Urzo AD, Gelb AF, Lakkis H, Garcia Gil E, Caracta CF. Efficacy and safety of a 12-week treatment with twice-daily Aclidinium bromide in COPD patients (ACCORD COPD I). COPD. 2012;9(2):90–101. doi:10.3109/15412555.2012.661492
21. Buhl R, Dunn LJ, Disdier C, et al. Blinded 12-week comparison of once-daily indacaterol and tiotropium in COPD. Eur Respir J. 2011;38(4):797–803. doi:10.1183/09031936.00191810
22. Seymour JM, Moore L, Jolley CJ, et al. Outpatient pulmonary rehabilitation following acute exacerbations of COPD. Thorax. 2010;65(5):423–428. doi:10.1136/thx.2009.124164
23. Stanojevic S, Kaminsky DA, Miller MR, et al. ERS/ATS technical standard on interpretive strategies for routine lung function tests. Eur Respir J. 2022;60(1):1.
24. Mannino DM, Buist AS, Petty TL, Enright PL, Redd SC. Lung function and mortality in the United States: data from the first national health and nutrition examination survey follow up study. Thorax. 2003;58(5):388–393. doi:10.1136/thorax.58.5.388
25. Doherty DE. A review of the role of FEV1 in the COPD paradigm. COPD. 2008;5(5):310–318. doi:10.1080/15412550802363386
26. Sciurba F, Criner GJ, Lee SM, et al. Six-minute walk distance in chronic obstructive pulmonary disease: reproducibility and effect of walking course layout and length. Am J Respir Crit Care Med. 2003;167(11):1522–1527. doi:10.1164/rccm.200203-166OC
27. Hernandes NA, Wouters EF, Meijer K, Annegarn J, Pitta F, Spruit MA. Reproducibility of 6-minute walking test in patients with COPD. Eur Respir J. 2011;38(2):261–267. doi:10.1183/09031936.00142010
28. Barnes PJ. Inflammatory mechanisms in patients with chronic obstructive pulmonary disease. J Allergy Clin Immunol. 2016;138(1):16–27. doi:10.1016/j.jaci.2016.05.011
29. Fermont JM, Masconi KL, Jensen MT, et al. Biomarkers and clinical outcomes in COPD: a systematic review and meta-analysis. Thorax. 2019;74(5):439. doi:10.1136/thoraxjnl-2018-211855
30. Bogaty P, Dagenais GR, Joseph L, et al. Time variability of C-reactive protein: implications for clinical risk stratification. PLoS One. 2013;8(4):e60759. doi:10.1371/journal.pone.0060759
31. Pinto-Plata VM, Müllerova H, Toso JF, et al. C-reactive protein in patients with COPD, control smokers and non-smokers. Thorax. 2006;61(1):23–28. doi:10.1136/thx.2005.042200
32. Khan MS, Butler J. Stability of changes in health status: next step in comprehensively assessing patient-reported outcomes. JAMA. 2022;328(10):923–924. doi:10.1001/jama.2022.13551
33. Partridge MR, Karlsson N, Small IR. Patient insight into the impact of chronic obstructive pulmonary disease in the morning: an internet survey. Curr Med Res Opin. 2009;25(8):2043–2048. doi:10.1185/03007990903103006
34. Kessler R, Partridge MR, Miravitlles M, et al. Symptom variability in patients with severe COPD: a pan-European cross-sectional study. Eur Respir J. 2011;37(2):264–272. doi:10.1183/09031936.00051110
35. Hansen H, Beyer N, Frølich A, Godtfredsen N, Bieler T. Inter-day test-retest reproducibility of the CAT, CCQ, Hads and EQ-5D-3L in patients with severe and very severe COPD. Patient Relat Outcome Meas. 2021;12:117–128. doi:10.2147/PROM.S306352
36. Incalzi RA, Pennazza G, Scarlata S, et al. Reproducibility and respiratory function correlates of exhaled breath fingerprint in chronic obstructive pulmonary disease. PLoS One. 2012;7(10):e45396. doi:10.1371/journal.pone.0045396
37. Munari AB, Silva I, Gulart AA, et al. Reproducibility of the 6-min step test in subjects with COPD. Respir Care. 2021;66(2):292–299. doi:10.4187/respcare.08096
38. Herpel LB, Kanner RE, Lee SM, et al. Variability of spirometry in chronic obstructive pulmonary disease: results from two clinical trials. Am J Respir Crit Care Med. 2006;173(10):1106–1113. doi:10.1164/rccm.200506-975OC