Background
Most cardiovascular risk prediction models use a medium-term time frame, most commonly 10 years. However, as age dominates cardiovascular risk, a potential problem of basing treatment on 10-year predicted risk is that younger people with very unfavourable risk factor profiles may not have high enough 10-year risk to be recommended for preventative treatment, even though in the longer-term their risk is very high.27,78–80 International guidelines recommend consideration of lifetime risk in younger people alongside 10-year risk, although NICE does not.10 The QRISK-Lifetime prediction tool was created in the same data set as QRISK2, and can be used as a standalone web-based tool81 as well as being the risk engine underlying the Joint British Societies’ risk calculator (JBS3)79 and Heart Age82 tools. Although lifetime tools are not currently recommended for CVD risk stratification by NICE,10 lifetime tools have been identified as a topic to examine further in a future guideline update.13
Unlike QRISK2 and QRISK3, QRISK-Lifetime does account for competing mortality risk, which is further potential advantage. However, lifetime models are difficult to validate, as observational data sets only very rarely observe events over a lifetime (e.g. some analyses using the Framingham study have almost 40 years of follow-up83). The QRISK-Lifetime model, therefore, uses information about older people in the data set to predict what will happen to younger people in the future. There is, therefore, an assumption that observed risk in older people now will apply to younger people in several decades time, and this is a very strong assumption given the large declines in age-standardised incidence of CVD in high-income countries since the 1970s, and the unknown effect of changing risk factors more recently (with declines in smoking in high-income countries, but increases in obesity, type 2 diabetes and sedentary behaviour). Similar issues (e.g. very long follow-up) apply to data sets like the Framingham study, as, by definition, patients enter the cohort in the distant past. Given available UK data, external validation is, therefore, in practice only possible over shorter time horizons. For example, internal validation of QRISK-Lifetime in the original derivation study examined predictive performance over a 10-year time horizon and compared with QRISK2, which predicts over the same time horizon.27 However, if tool performance is poorly calibrated in different age groups, then it is possible to infer performance over a lifetime. Although there have been studies of reclassification using QRISK-Lifetime-predicted lifetime risk compared with QRISK3 10-year predicted risk,78 to our knowledge there has not been an independent external validation.
This chapter reports findings in relation to objective 3 (i.e. to externally validate the QRISK-Lifetime risk prediction tool for primary prevention of CVD), examining discrimination and calibration over a 10-year time horizon and the characteristics of those reclassified as high risk using QRISK-Lifetime rather than QRISK3.
Methods
Data sources, outcome definition, other variable definitions and missing data
The same data set used for QRISK3 external validation was used (see Chapter 2, Methods), with the same variation in methods applying (i.e. a cohort entry date of 1 January 2004 vs. 1 January 1998 for QRISK-Lifetime, our implementation did not allow the use of future cholesterol values and Townsend deprivation scores were fitted as the median of vigintiles of Townsend score).
Analytical methods
The lifetime (i.e. to age 95 years) and 10-year risk of experiencing a cardiovascular event was calculated for each patient using publicly available QRISK-Lifetime 2011 software (under GNU Lesser General Public Licence version 3) without recalibration. As lifetime risk is not observed in the validation data set, the performance of the risk score was assessed by examining discrimination and calibration of the model using the same methods as for QRISK3 external validation (see Chapter 2, Methods) over a 10-year time horizon, as was carried out in the original derivation and internal validation study.27 For both men and women, calibration was evaluated in the whole population and in prespecified subgroups of age and mCCI. Calibration refers to how closely predicted risk and observed probabilities agree at group level. As QRISK-Lifetime accounts for competing mortality risk, we evaluated calibration using only the Aalen–Johansen estimator of observed risk in censored survival data (i.e. an extension of the Kaplan–Meier estimator, which allows for competing events, non-CVD death in this case).84
Clinical guideline recommendations for primary preventative treatment of CVD classify patients in relation to thresholds of predicted risk. In England and Wales, NICE-recommend treatment if 10-year predicted CVD risk is ≥ 10%. Consistent with the validation of QRISK-Lifetime over a 10-year time horizon, we examined changes in which patients were recommended for treatment based on either a QRISK3 or QRISK-Lifetime 10-year predicted risk of ≥ 10%.
However, there is no recommended threshold of lifetime risk at which to offer treatment,10 although NICE has signalled that consideration of lifetime risk models is of interest in any future guideline update.13 Therefore, we additionally calculated the proportion of men and women recommended for treatment by QRISK3 at the 10% threshold, and used a cut-off point of QRISK-Lifetime-predicted lifetime risk above which exactly the same proportion of participants lay.
For both comparisons (i.e. QRISK3 and QRISK-Lifetime 10-year prediction > 10%; and QRISK3 prediction > 10% and matched number of participants with the highest lifetime predicted risk), we examined the characteristics of patients recommended for treatment, the observed risk of CVD at 10 years, and the number needed to treat (NNT) to prevent one new CVD event assuming all people recommended for treatment actually took a statin and with a relative risk reduction of 25% for new CVD events. All models were fitted in R and Stata.
Results 1: external validation of QRISK-Lifetime
There were 1,260,329 women and 1,223,265 men aged 30–84 years in the external validation data set, with a mean age of 49.3 years for women and 47.6 years for men (see Appendix 2, Table 31). Baseline characteristics were similar to QRISK-Lifetime internal validation, with the exception of the external validation data set having somewhat fewer people with a recorded family history of premature CVD and somewhat more people with treated hypertension and CKD.
Evaluated at 10 years’ follow-up, QRISK-Lifetime had excellent discrimination in the whole population of both women (Harrell’s c-statistic 0.844, 95% CI 0.841 to 0.847) and men (Harrell’s c-statistic 0.808, 95% CI 0.806 to 0.811), similar to the QRISK-Lifetime internal validation [area under the receiver operating curve (AUROC): women, 0.842; men, 0.828] ().27 Explained variation in women was 53.3% compared with 45.5% in men. Stratified by age, discrimination varied from good in people aged 30–44 years (Harrell’s c-statistic: women, 0.714; men, 0.714) to poor in people aged 75–84 years (Harrell’s c-statistic: women, 0.578; men, 0.556), and explained variation progressively declined with age. Stratified by comorbidity, discrimination varied from excellent in people with low comorbidity (mCCI = 0 Harrell’s c-statistic: women, 0.844; men, 0.803) to moderate to good in people with high comorbidity (mCCI ≥ 3 Harrell’s c-statistic: women, 0.724; men, 0.656).
Discrimination of QRISK-Lifetime (evaluated at 10 years)
In the whole population, calibration was reasonable at lower levels of predicted risk for both men and women, but there was considerable overprediction at higher levels of predicted risk (). Stratified by age, in both men and women, there was some underprediction in people aged 30–44 years, calibration was good in people aged 45–64 years and there was considerable underprediction in older people (see ). Stratified by comorbidity, in both men and women, there was underprediction at higher levels of predicted risk in people with low comorbidity (mCCI = 0) and consistent underprediction in people with higher comorbidity (see ).
Calibration of QRISK-Lifetime in women and men (evaluated at 10 years): whole population and stratified by age group and CCI score. (a) Women: calibration in whole population; (b) men: calibration in whole population; (c) women: calibration by age group; (more...)
Results 2: reclassification of study participants by QRISK-Lifetime compared with QRISK3
Reclassification using QRISK-Lifetime constrained to 10-year versus QRISK3 10-year prediction
In the first reclassification analysis examining people with 10-year predicted CVD risk > 10% using QRISK-Lifetime and QRISK3, QRISK-Lifetime classified fewer people as eligible to be offered a statin than QRISK3 (). QRISK-Lifetime classified 194,411 (15.4%) women as high risk, compared with 239,396 (19.0%) women classified as high risk by QRISK3. QRISK-Lifetime classified 276,369 (22.6%) men as high risk, compared with 341,962 (28.0%) men classified as high risk by QRISK3. For women, 15.1% were classified as high risk by both tools, 3.9% were classified as high risk by QRISK3 only and 0.3% were classified as high risk by QRISK-Lifetime only, over 10 years. For men, 21.9% were classified as high risk by both tools, 6.1% were classified as high risk by QRISK3 only and 0.7% were classified as high risk by QRISK-Lifetime only, over 10 years.
Reclassification between QRISK3 and QRISK-Lifetime based on 10-year risk prediction for both
Based on 10-year risk prediction, the characteristics of people classified as high risk by each tool were similar (). Fewer people were recommended for treatment by QRISK-Lifetime and there were fewer observed events in people recommended for treatment by QRISK-Lifetime (women, 25,461 vs. 28,373; men, 33,450 vs. 37,026), but the percentage of people experiencing an event was higher (women, 13.2% vs. 11.9%; men, 12.1% vs. 10.8%). Among people recommended for treatment with a statin, the estimated NNT from statin prescription to prevent one event was 30 and 34 in women, and 33 and 37 in men, for QRISK-Lifetime and QRISK3, respectively (see ).
Characteristics of those recommended for treatment by QRISK3 10-year prediction, QRISK-Lifetime 10-year prediction and QRISK-Lifetime lifetime prediction (matched numbers of patients to QRISK3)
Reclassification using QRISK-Lifetime lifetime risk versus QRISK3 10-year prediction
By design, the comparison with QRISK-Lifetime predicting to age 95 years (i.e. lifetime risk) was constrained to include 19.0% of women and 28.0% of men at the highest predicted lifetime risk (i.e. the same proportion of people identified as high risk based on QRISK3 10-year prediction) (see ). Only 5.3% of women were identified as high risk by both QRISK3 and QRISK-Lifetime predicting to age 95 years, with a different 13.7% of women identified as high risk by one or other of the prediction tools. For men, 8.9% were identified as high risk by both prediction tools and a different 19.1% of men by one or other of the tools.
Compared with people identified as high risk by QRISK3 10-year prediction, people with the highest predicted lifetime risk were much younger, had a lower mean SBP and had a somewhat higher mean TC : HDL and BMI. In addition, a lower proportion of people with the highest predicted lifetime risk had treated hypertension and a much higher proportion had a family history of premature CVD and were from a minority ethnic background (see ). Compared with people recommended for treatment based on 10-year predicted risk, there were fewer CVD events observed in people at the highest predicted lifetime risk [women, 9652 (4.0%) vs. 28,373 (11.9%); men, 14,725 (4.3%) vs. 37,026 (10.8%)]. For QRISK-Lifetime predicting to age 95 years, the estimated NNT to prevent one CVD event from statin treatment was 99 and 100 in women and men, respectively, compared with 34 and 37 in women and men, respectively, among those with > 10% 10-year QRISK3-predicted risk (see ).
Summary
QRISK-Lifetime external validation
It is essentially impossible to validate lifetime risk models because lifetime follow-up of observed events either is not available or would be so historical that it would be a poor guide to performance of the model in a contemporary population.60,85 Similar to the original internal validation,27 we evaluated QRISK-Lifetime over a 10-year prediction horizon. Within this constraint, QRISK-Lifetime had excellent discrimination in the whole population; however, as with QRISK3, discrimination was poor to moderate within the age strata, and moderate within the comorbidity strata. Calibration plots showed some underprediction in the whole population, with large underprediction in older people and people with multimorbidity at higher levels of predicted risk.
Comparing people recommended for treatment at a 10-year 10% risk threshold, QRISK-Lifetime (predicting over 10 years) recommended fewer people for statin treatment (i.e. 15.4% of women and 22.6% of men) than QRISK3 (i.e. 19.0% of women and 28.0% of men), although the people recommended experienced slightly more CVD events and the estimated NNT to prevent one CVD event was slightly lower for QRISK-Lifetime. Characteristics of people recommended for treatment over a 10-year prediction horizon were broadly similar.
Comparing people recommended for treatment by QRISK3-predicted 10-year risk ≥ 10% compared with the same proportion at highest estimated lifetime risk by QRISK-Lifetime, there was only a small overlap between the populations at highest predicted risk by the different tools. By design, each tool ‘recommended’ 19.0% of women and 28.0% of men for treatment. Only 5.3% of women and 8.9% of men were recommended for treatment by both tools, and the people recommended for treatment were considerably different, as other studies have found.27,78 People with highest predicted lifetime risk were considerably younger, were more likely to have a family history of premature CVD and were more likely to be from a minority ethnic background. Over 10 years, people with highest predicted lifetime risk experienced many fewer CVD events (as expected given age differences), with people at highest predicted lifetime risk having an estimated NNT (with a statin to prevent one CVD event) approximately three times larger than for people recommended for treatment by QRISK3 (in women, QRISK-Lifetime highest risk NNT = 99 vs. QRISK3 risk > 10% NNT = 34; in men, QRISK-Lifetime highest risk NNT = 100 vs. QRISK3 risk > 10% NNT = 37). There is, therefore, a considerable leap of faith involved in treating based on lifetime risk, as the medium-term (10-year) benefit is considerably lower.
Limitations
This study has the same limitations as those already described in Chapter 2, Limitations, for any routine data study, but two particular limitations specifically apply. First, as with previous studies,35,67 we used multiple imputation, but the assumption that data are missing at random may be incorrect, and this is probably more likely to apply in younger people (i.e. the key target population for lifetime CVD risk prediction), as CVD risk assessment (particularly measurement of cholesterol) is likely to be carried out in people who are already suspected to be high risk.35 Second, evaluating lifetime risk in a study with relatively short follow-up is intrinsically problematic. In this study, median follow-up of study participants was 5.7 years (interquartile range 2.2–10.2) years in women and 5.2 (interquartile range 2.0–9.3) years in men. The QRISK-Lifetime derivation paper does not state follow-up time,27 but follow-up time in this study is similar to follow-up in QRISK2 derivation, which uses a similar data set to QRISK-Lifetime.86 Lifetime risk is, therefore, being estimated and evaluated from relatively short periods of observation. However, lifetime risk is estimated by assuming that future risk beyond the period of observation will be the same as that observed for older people during the period of observation. As QRISK-Lifetime systematically underpredicts CVD risk over the period of observation, then lifetime estimates must also underpredict. More generally, however, it is a very strong assumption that age-specific CVD incidence will be stable over the next few decades, given falling CVD incidence over the last few decades and large increases in obesity, sedentary behaviour and diabetes in the last two decades.
