Background: Contemporary management of chronic obstructive pulmonary disease (COPD) relies on exacerbation history to risk-stratify patients for future exacerbations. Multivariable prediction models can improve the performance of risk stratification. However, the clinical utility of risk stratification can vary from one population to another.

Research question: How do two validated exacerbation risk prediction models (ACCEPT and Bertens) compared to exacerbation history alone perform in different patient populations?

Study design and methods: We used data from three clinical studies representing populations at different levels of moderate/severe exacerbation risk: the Study to Understand Mortality and Morbidity in COPD (SUMMIT, N=2,421, annual risk=0.22), the Long-term Oxygen Treatment Trial (LOTT, N=595, annual risk=0.38), and Towards a Revolution in COPD Health (TORCH, N=1,091, annual risk=0.52). We compared the area under the curve (AUC) and net benefit (measure of clinical utility) among three risk stratification algorithms for predicting exacerbations in the next 12 months. We also evaluated the effect of model recalibration on clinical utility.

Results: Compared to exacerbation history, ACCEPT had better performance in all three samples (ΔAUC 0.08, 0.07, and 0.10 - all P-values ≤0.001). Bertens had better performance compared to exacerbation history in SUMMIT and TORCH (ΔAUC 0.10 and 0.05 - P-values <0.001), but not in LOTT. No algorithm was superior in clinical utility across all samples. Before recalibration, Bertens generally outperformed the other algorithms in low-risk settings while ACCEPT outperformed others in high-risk settings. All three algorithms had the risk of harm (providing lower net benefit than not using any risk stratification). After recalibration, risk of harm was substantially mitigated for both prediction models.

Interpretation: Exacerbation history alone is unlikely to provide clinical utility for predicting COPD exacerbations in all settings and could be associated with a risk of harm. Prediction models have superior predictive performance but require setting-specific recalibration to confer higher clinical utility.

Keywords: Chronic obstructive pulmonary disease; Clinical prediction modeling; Clinical utility; Precision medicine; Risk stratification.