Surgical risk stratification tools
Surgical RST, used to predict perioperative morbidity and mortality, fall under two general categories: physiologic scores and risk prediction models. These tools are explicitly designed for use in the perioperative setting. The applicability of some has expanded to the realm of quality evaluation and can form a scaffold for evidence based recommendations in practice guidelines.
American Society of Anesthesiologists Physical Status Grading
The American Society of Anesthesiologists Physical Status Grading (ASA-PS) was introduced in 1941 as a preoperative grading of the physical status of patients, and was subsequently modified in 1963.66 67 There are five grades based on the presence or absence of mild to serious life-threatening systemic disease, with an additional classification of E designated for emergency surgery. Higher ASA-PS grades have been correlated with prolonged postoperative in-hospital stay, postoperative surgical ICU admission and the development of serious postoperative sepsis, although studies have also demonstrated overestimates of mortality using this grading system.68–70
ASA-PS has generally been shown to be an acceptable predictor of perioperative mortality and by design is applicable for use early in patient evaluation, can be incorporated into audits and facilities clinical practice. Its specific applicability in the EGS patient population has not been examined, and while the score is based on factors that may be available in a patients chart, there is some subjectivity to this scores tabulation, leading to interobserver variability.
Criteria met: 3, 4, 5, 6
Criteria not met: 1, 2
ACS-NSQIP universal surgical risk calculator
The American College of Surgeons National Surgical Quality Improvement Programme (ACS-NSQIP) database was used to develop a web-based tool to estimate the surgical risk of most operations. The resulting Universal Surgical Risk Calculator tool requires 21 preoperative factors (demographics, comorbidities, procedure), using regression models to predict mortality as well as six postoperative complications based on the preoperative risk factors.3 This surgical risk calculator has good discrimination and calibration and has only slight differences from previous procedure-specific ACS-NSQIP calculators. The ACS-NSQIP Universal Surgical Risk Calculator has been externally validated in an EGS population and was found to have a slight underestimation of the risk of EGS compared with the risk of elective surgery. While these differences were statistically significant, they were small with an observed to expected mortality ratio of 1.03 for EGS, leading the authors to conclude that it is applicable to emergency as well as elective patients.71 It is much more comprehensive than other risk calculators and has a web application which can be easily used.72
Overall, the ACS-NSQIP Universal Surgical Risk Calculator meets our specified criteria. It is reasonably accurate in terms of estimation of postoperative death and complication in EGS, uses readily obtainable objective data, can be used in early phases of care and can be used for auditing purposes. Its calculation requires access to an internet-based portal, which generally is facilitated in clinical practice, but may limit its use in certain environments. One notable fact, however, is that the ACS-NSQIP Universal Surgical Risk Calculator requires a CPT code for a specific operation, and therefore is not applicable to non-operative EGS patients.
Criteria met: 1, 2, 3, 5, 6
Criteria not met: 4
Charlson Comorbidity Index
The Charlson Comorbidity Index (CCI) was originally developed to facilitate classification of risk of death extrapolated from comorbidities, to be used in longitudinal studies.73 It was developed using 1 year mortality data in patients admitted to the medical service at New York Hospital and validated in patients with primary breast cancer at Yale New Haven Hospital over a 10-year follow-up period. Nineteen comorbid conditions were classified and a weighted index was developed that accounted for the number and seriousness of comorbid diseases.74 This was later modified by Charlson to add age to the comorbidity index, formulating the Charlson Age-Comorbidity Index (CACI). The CACI combines 19 medical conditions weighted 1–6, with age weighted 1 for every decade past 40 years.74
The CACI has been validated in the setting of medical patients, critically ill and trauma patients as well as many elective surgical settings. Evidence to evaluate the role of CCI and CACI in the emergency setting is sparse. A large review of Saudi Arabian patients demonstrated CCI as a reliable comorbidity index for prediction outcomes in acute non-traumatic surgical patients.75 An Australian study of 2819 patients demonstrated a correlation between the CCI as a measure of preoperative comorbidity and postoperative mortality in trauma patients.76 One study of 257 EGS patients undergoing an operation found that increasing CACI scores were associated with higher rates of 30-day postoperative mortality.77
Despite a lack of incorporation of explicitly surgical factors, the literature suggests that CCI and CACI may accurately estimate morbidity and mortality in the EGS population. These scores are based on a broad range of comorbid conditions and objective data points, can be applied early and can be used for auditing. The limitation to the use of these scores in EGS falls in their sometimes cumbersome calculation; with so many factors to account for, they are not well suited to the time sensitive preoperative environment.
Criteria met: 1, 2, 3, 4, 5
Criteria not met: 6
Emergency Surgery Acuity Score
The Emergency Surgery Acuity Score is a preoperative risk stratification system that predicts perioperative mortality in emergency surgery patients.7 It captures both patient comorbidities and the acute physiology at presentation. This score was formed by multiple logistic regression analyses of independent predictors of mortality based on preoperative variables in ACS-NSQIP data fields. The ESAS includes 3 demographic variables, 10 comorbidities and 9 laboratory variables. Based on the relative impact of these 22 predictors, using weighted averages, a score is derived that ranges from 0 to 29 points. The observed probability of 30-day mortality is 0% at a score of 0, 36% at a score of 11, 75% at a score of 19% and 100% for scores of 22 or greater. The ESAS has been validated using the 2012 ACS-NSQIP database and has demonstrated a higher discriminatory power in comparison with ASA-PS.
ESAS has demonstrated the ability to differentiate mortality accurately, although our understanding of its use may benefit from prospective examination. Further work is needed to understand the applicability of this score to EGS postoperative morbidity. It is based entirely on objective factors that can be easily identified preoperatively. The score could be standardized to be used for auditing and its straightforward calculation model renders it easy to apply clinically, although no ready-to-use application exists as of yet. While this risk score could be used in non-operative patients as it does not require any surgical variables, it was derived from NSQIP data which only contains operative cases. Therefore, as it was derived and validated using only operative patients, further validation will be necessary for this score to be reliably applied to non-operative EGS patients.
Criteria met: 1, 2, 3, 5
Criteria not met: 4, 6
Physiological Emergency Surgery Acuity Score
Like ESAS, the Physiological Emergency Surgery Acuity Score (PESAS), was derived from sequential analyses of ACS-NSQIP data, this time focused on information collected in 2011. The resulting score is composed of 10 physiologic points linked to advanced clinical acuity and postoperative mortality.78 PESAS was subsequently internally validated on ACS-NSQIP data from 2012, yielding a c-statistic of 0.79 (95% CI receiver operating characteristic 0.7801 to 0.8025).79
This score was explicitly created for use in the EGS patient population and is designed to assess operative risk of mortality using objective data in a time-sensitive, applicable manner. However, this score may not fully capture the extent of a patient’s coexisting conditions and is not validated to assess morbidity. This score can be applied early in a patients care trajectory, can be used for auditing purposes and is appropriate for clinical use. As discussed with ESAS, as this score was derived and validated using operative patients, further validation is needed for reliable use with non-operative EGS patients.
Criteria met: 1, 2, 3, 5
Criteria not met: 4, 6
Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity
The Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity (POSSUM) was devised in 1996 in the UK as a simple scoring system that could be used in surgical audit.80 The original POSSUM is a 12 factor, four grade, physiological score (ranging from 12 through 88) plus a 6-factor operative severity score (ranging from 6 through 44). These two scores are inserted into two formulas to give predicted morbidity and mortality. Correlation in predicted versus observed rates of mortality and morbidity has been demonstrated.80 However, the POSSUM overestimates mortality rates, especially in low-risk procedures. The lowest possible scores that POSSUM allows are 12 for physiology and 6 for operative severity, which predicts the lowest risk of death as 1.08%.80
When Whitely et al
81 applied the POSSUM in Portsmouth, UK hospitals, it was found that the POSSUM overpredicted the total number of deaths by a factor of 2, performing worst in low-risk patients (by a factor of 6 in those with predicted risk of death of 10% or less). Using these data, their team remodeled the tool, creating a Portsmouth predictor equation for mortality. This Portsmouth POSSUM (P-POSSUM) still overpredicts death in the low-risk patient with the lowest predictive risk for death of 0.20%, which is still higher than expected for patients with no physiologic abnormalities and age <61 years.81 More recently, web-based calculators have become available, making this scoring system more user-friendly.
The limited data on use of POSSUM scores in EGS patients have demonstrated overestimates of mortality rates for low-risk procedures. As this score is based partially on operative data, it cannot be used in the preoperative setting and will not be useful for non-operative cases. Some data points, such as chest radiograph interpretation, are subjective in nature. This may limit its use for auditing purposes and its generalized use in modern EGS practice.
Criteria met: 5, 6
Criteria not met: 1, 2, 3, 4
Perioperative Mortality Risk Score
The Perioperative Mortality Risk Score (PMRS) is a risk score for 30-day postoperative mortality derived from a study of surgical patients aged 70 years or older from three large metropolitan teaching hospitals.82 Risk factors for 30-day mortality were determined by multivariate analysis and the risk score for each of six factors were summed to give an overall PMRS. The score was initially validated using 256 patients from a regional hospital.
While appropriate for the elderly EGS subset, this score is not generalizable across the entirety of the EGS population. It is based on readily available preoperative and postoperative data, although obviously half of these factors cannot be evaluated in the preoperative period. The lack of overall applicability of this score may limit its utility in audits. The calculation is not complex and can be applied at the bedside.
Criteria met: 2, 6
Criteria not met: 1, 3, 4, 5
Surgical Apgar Score
The Surgical Apgar Score (SAS) is a 10-point score based on the lowest heart rate, lowest mean arterial pressure and estimated blood loss intraoperatively.1 It has been validated to discriminate well between groups of patients at high and low risk of major complications and death within 30 days of operation after colectomy and general/vascular surgery. However, original studies of this score included only 3%–6% emergency operations. Further study has suggested that while the SAS has strong predictive value for postoperative morbidity, it is weakly discriminative and thus should not be used as a standalone tool for evaluation.66 Because the variables are influenced by the performance of the medical teams and the patient’s earlier conditions, SAS does not allow comparison of quality between institutions or practitioners.
The SAS direct applicability to risk evaluation in EGS has not been adequately studied. It is an operatively focused score based on attained objective data points and therefore cannot be used in the preoperative setting. It is not appropriate for non-operative EGS patients. It can be readily applied following completion of the surgical procedure. However, it is not appropriate for auditing purposes as it reflects intraoperative care, not solely patient risk.
Criteria met: 2, 6
Criteria not met: 1, 3, 4, 5
Surgical Mortality Score
The original Surgical Mortality Score (SMS) is a risk stratification model for in-hospital mortality for patients undergoing surgical procedures developed across a range of surgical specialties in England.83 The model uses data from an existing administrative database: the hospital Patient Administration System at a London university hospital. SMS risk factors for in-hospital mortality include surgical specialty, age at operation, gender, mode of surgery (emergency or elective), time of onset of surgical procedure and median operating time. The SMS model is used to calculate the mortality risk ratio for each specialty, giving an SMS predicted in-hospital mortality that can be compared with the observed in-hospital mortality. The exact binomial 95% CI for the observed in-hospital mortality rate is then divided by the SMS-predicted risk to obtain the 95% CI of the mortality risk ratio.
This score has not been explicitly studied in the EGS population; its ability to accurately estimate morbidity and mortality for EGS is unknown. It is based on available data points, a portion of which are extracted from operating room chart review, which limits preoperative applicability and relevance for auditing. Its calculation is not entirely straightforward, which can render it cumbersome for generalized practice.
Criteria met: 2
Criteria not met: 1, 3, 4, 5, 6
Surgical Outcome Risk Tool
The Surgical Outcome Risk Tool (SORT) is a preoperative risk prediction tool for death within 30 days of surgery in adults undergoing non-cardiac, non-neurologic inpatient surgery.84 It was developed in 2014 in the UK. The six variables included are ASA-PS grade (III, IV and V), urgency of surgery (expedited, urgent, immediate), specialty, severity of surgery, presence of cancer and age. Model coefficients were determined and used to develop a formula for a risk score. Internal validation in the UK demonstrated the SORT to be more accurate than the ASA-PS and the SRS. A web-based calculator is available as well as an app to aid in calculations of the SORT.85
Although it does account for emergent case status, SORT tool has not been explicitly studied in the EGS patient population, thus its capacity to accurately estimate morbidity and mortality risk in this patient subset is unknown. Its inclusion of ASA scoring renders its calculation susceptible to some degree of subjectivity. This score is explicitly designed for use in the preoperative setting and generally can be used for auditing. Calculation of this score is relatively straightforward and, if appropriately validated, could be well suited for use in clinical practice.
Criteria met: 3, 5, 6
Criteria not met: 1, 2, 4
Surgical Risk Scale
The Surgical Risk Scale85 was developed as an alternative system for comparative audit that used easily collected clinical data independent of the surgeon. The Surgical Risk Scale has three components: the Confidential Enquiry into Perioperative Deaths grade (categorized as elective, scheduled, urgent and emergency), the ASA-PS grade and the British United Provident Association (BUPA) operative grade (categorized as major, intermediate, major, major plus and complex major).
This tool has not been explicitly studied for risk prediction in EGS and the Surgical Risk Scale has only been validated to predict death in single-center studies in the UK. The scoring is somewhat subjective, through incorporation of ASA status and BUPA categorization. Its use requires some familiarity with BUPA categories, which may not be readily available. This score can be applied early in a patients care course but may not be ideal for auditing given possible variances in interpretation of ASA and BUPA scoring. It also obviously cannot be used in non-operative patients. Although not a complex set of calculations, subjectivity and lack of familiarity with BUPA may limit the clinical relevance of this score.
Criteria met: 3
Criteria not met: 1, 2, 4, 5, 6
Surgical Risk Score
The Surgical Risk Score is an ASA-PS-based model to predict mortality based on preoperative data alone.86 It was formulated using data of all types of operations excluding cardiac surgeries and caesarian section. This score includes ASA-PS status, age, type of surgery (elective, urgent emergency) and degree of surgery (minor, moderate or major as described by the modified Johns Hopkins surgical severity score).
Like the Surgical Risk Scale, this tool has not been explicitly studied for risk prediction in EGS. Its incorporation of ASA status and Johns Hopkins Surgical Severity Score introduces a degree of subjectivity. The score can be applied early in a patients care and may be clinically helpful, but more work would be needed to describe its applicability to this patient subset. It also cannot be used in non-operative patients.
Criteria met: 3
Criteria not met: 1, 2, 4, 5, 6