The provision of high-quality care is a fundamental objective of the NHS.1 The most widely used framework for assessing the quality of health care is that proposed by Donabedian.2 He describes three distinct domains from which inferences about the quality of care can be made: structure, process and outcome. Structure describes the attributes of the setting within which care occurs. Process denotes what is actually done in giving and receiving care. Outcome is defined as the effects of care on the health status of patients and populations.
The domains of structure and process can be assessed within an individual health-care provider. Indicators of structure are assessed against professional standards, regulations and recommendations. They relate to the provider rather than the patient and, therefore, require only periodic assessment. Indicators of process are assessed against national or international clinical guidelines, based ideally on high-quality evidence. The gold standard is 100% adherence to the guidelines and there is no need to compare performance with other health-care providers.
Assessing outcomes for a single health-care provider requires comparison against other providers (comparative audit) to put the outcome of the particular health-care provider in context and to enable benchmarking. However, the quality of care is only one of many factors that will contribute to a patient’s outcome and, if crude outcomes were to be compared between health-care providers, then any effect of quality would probably be overwhelmed by variation in the patient demographics, underlying health status, acute conditions and severity of the acute illness (factors collectively termed ‘case mix’). When comparing outcomes between health-care providers, it is therefore essential to take the differing case mix of patients treated by the providers into account in order to be able to make fair comparisons.
National clinical audit has a key role to play in ensuring high-quality care,3,4 particularly in areas of health care, such as emergency and critical care, where patient choice does not, and cannot, play a significant part. Sophisticated and accurate risk prediction models, developed using high-quality clinical data, are key in underpinning fair comparisons among health-care providers.5 They can also enable risk-adjusted observational research and risk stratification in randomised controlled trials.
The Intensive Care National Audit & Research Centre (ICNARC) is an independent charitable organisation that runs national clinical audit programmes to monitor and improve care for the critically ill. ICNARC co-ordinates two national clinical audits: the Case Mix Programme (CMP), a national clinical audit for adult critical care; and the National Cardiac Arrest Audit (NCAA), a national clinical audit for in-hospital cardiac arrest, co-ordinated jointly with the Resuscitation Council (UK). Both national clinical audits are listed for inclusion in the Department of Health’s NHS Quality Accounts,6 and both are underpinned by the need and ability to report accurate risk-adjusted outcomes.
The Case Mix Programme
The CMP is the national clinical audit for adult critical care, with a remit for England, Wales and Northern Ireland. Participation of adult, general critical care units delivering level 3 or combined level 2/3 care (intensive care units and combined high-dependency/intensive care units) is approaching 100%, but participation of other critical care units, such as specialist units (e.g. neurocritical care units and cardiothoracic critical care units) and stand-alone level 2 (high-dependency) units, is lower. For all participating units, data on consecutive admissions are recorded prospectively and abstracted from the medical records by trained data collectors in accordance with precise rules and definitions. The data collected include raw physiological and diagnostic data from the first 24 hours following admission to the critical care unit, together with demographic, outcome and activity data. The data undergo extensive validation, both locally and centrally, before being pooled into the central CMP database. Details of data collection and validation have been reported previously, and the CMP database has been independently assessed to be of high quality.7
Risk prediction models for adult, general critical care are well established, but ongoing improvement work is essential to further improve accuracy.8 In 2006, ICNARC published a validation of four existing models [the Acute Physiology and Chronic Health Evaluation (APACHE) model versions II and III,9,10 the Simplified Acute Physiology Score (SAPS) version II11 and the Mortality Probability Models (MPM) version II12] and concluded that there was little difference in performance among the models, but that there was scope for further improvement.13 While retaining the APACHE II model for the purpose of international comparisons, ICNARC developed and validated the ICNARC model,14,15 which underpins the risk-adjusted outcomes reported for the CMP. However, a number of areas were identified where we have the potential to improve our modelling.
The National Cardiac Arrest Audit
The NCAA is the national clinical audit of patients, aged greater than 28 days, in acute hospitals in the UK who receive cardiopulmonary resuscitation (CPR) and are attended by the hospital-based resuscitation team (or equivalent) in response to a 2222 call (2222 is the emergency telephone number used to summon a resuscitation team in UK hospitals). CPR is defined in the NCAA as chest compressions and/or defibrillation. Standardised data are collected at the time of the cardiac arrest and from the medical records in accordance with precise rules and definitions. Staff members at participating hospitals enter data onto a dedicated secure online data entry system. Data are validated, both at the point of entry and centrally, for completeness, illogicalities and inconsistencies. Details of data collection and validation have been reported previously.16
Prior to this project, there was no validated risk prediction model for predicting outcomes following in-hospital cardiac arrest. Initial comparative reporting for the NCAA was based on stratifying patients according to single risk factors.
Governance
Both the CMP and NCAA have approval from the Confidentiality Advisory Group of the Health Research Authority for the collection and use of patient-identifiable data without consent under Section 251 of the NHS Act 200617 [approval numbers PIAG 2-10(f)/2005 and ECC 2-06(n)/2009].
The project was overseen by an Expert Advisory Group (see Acknowledgements), which included a member who had previous experience as a patient in a critical care unit. The Expert Advisory Group met five times during the project, reviewed its progress against each of the objectives and gave advice on the future directions for the project. Individual members of the Expert Advisory Group were also contacted between meetings and asked to provide input in their particular areas of expertise.
Aim and objectives
The aim of the current project was to improve risk prediction models to underpin quality improvement programmes for the critically ill (patients receiving general or specialist adult critical care or experiencing an in-hospital cardiac arrest).
We set out to address this aim through the following objectives:
To improve current risk prediction models for critically ill patients, to include:
external validation of current models in critical care units in Scotland
introduction of new important variables
improved modelling of interactions between physiological parameters
improved handling of missing data and
improved modelling of reasons for admission to/diagnosis on admission to critical care.
To develop and validate new risk prediction models for critically ill patients, to include:
models for cardiothoracic critical care
models for patients experiencing an in-hospital cardiac arrest and
models for critical care units admitting lower-risk patients.
Immediate translation of improved risk prediction models into practice, through:
adoption into routine comparative outcome reporting for national clinical audits and
communication of research output to providers, managers, commissioners, policy-makers and academics in critical care.
Following early advice from the Expert Advisory Group, objective 2iii was incorporated into objective 1 by aiming to improve risk prediction at the patient level to the extent that a separate model for critical care units admitting lower-risk patients was no longer required.
Chapter 2 reports the external validation of the current ICNARC model in critical care units in Scotland (objective 1i). Chapter 3 reports the development and validation of a preliminary risk prediction model for admissions to specialist cardiothoracic critical care units (objective 2i). Chapter 4 reports the development and validation of a new, improved risk prediction model for admissions to all adult (general and specialist) critical care units (objectives 1ii–v). Chapter 5 reports the development and validation of risk prediction models for outcomes following in-hospital cardiac arrest (objective 2ii). Chapter 6 reports the translation and dissemination work conducted to date (objective 3). Chapter 7 draws conclusions from the project as a whole, including implications for practice, and makes recommendations for further research in this field.
