You are here

Article Text

Article menu

Download PDFPDF

Review
Risk stratification tools in emergency general surgery
  1. Joaquim Michael Havens1,2,
  2. Alexandra B Columbus2,
  3. Anupamaa J Seshadri1,
  4. Carlos V R Brown3,
  5. Gail T Tominaga4,
  6. Nathan T Mowery5,
  7. Marie Crandall6
  1. 1 Division of Trauma, Burns and Surgical Critical Care, Brigham and Women’s Hospital, Boston, Massachusetts, USA
  2. 2 Center for Surgery and Public Health, Brigham and Women’s Hospital, Boston, Massachusetts, USA
  3. 3 Division of Acute Care Surgery, Dell Medical School, University of Texas at Austin, Austin, Texas, USA
  4. 4 Department of Surgery, Scripps Memorial Hospital La Jolla, La Jolla, California, USA
  5. 5 Department of Surgery, Wake Forest Baptist Medical Center, Winston-Salem, North Carolina, USA
  6. 6 Department of Surgery, University of Florida College of Medicine, Jacksonville, Florida, USA
  1. Correspondence to Dr Joaquim Michael Havens, Division of Trauma, Burns and Surgical Critical Care, Brigham and Women’s Hospital, Boston, MA 02115, USA; jhavens{at}bwh.harvard.edu

Abstract

The use of risk stratification tools (RST) aids in clinical triage, decision making and quality assessment in a wide variety of medical fields. Although emergency general surgery (EGS) is characterized by a comorbid, physiologically acute patient population with disparately high rates of perioperative morbidity and mortality, few RST have been explicitly examined in this setting. We examined the available RST with the intent of identifying a tool that comprehensively reflects an EGS patients perioperative risk for death or complication. The ideal tool would combine individualized assessment with relative ease of use. Trauma Scoring Systems, Critical Care Scoring Systems, Surgical Scoring Systems and Track and Trigger Models are reviewed here, with the conclusion that Emergency Surgery Acuity Score and the American College of Surgeons National Surgical Quality Improvement Programme Universal Surgical Risk Calculator are the most applicable and appropriate for EGS.

  • emergency general surgery
  • risk adjustment

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/tsaco-2017-000160

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Risk stratification tools (RST) facilitate a meaningful comparison of surgical outcomes between surgeons, hospitals and healthcare systems. These population-level comparisons form one basic measurement of quality of surgical care. In the perioperative setting, RST can help to objectify the clinical triage process and to quantify probability of serious morbidity and mortality.1–4 Such tools can thus support surgical decision making and can aid in the informed consent process.1 2 5

    Multiple methodologies have been used to create RST, ranging from expert consensus to logistic regression analysis.2 4 6 From these methods, probabilistic models are derived through statistical regression and are designed to predict risk within groups of patients, as opposed to deterministic models that are designed to predict risk for individuals. With the appropriate application of decision rules, however, they can be useful in aiding decision support between the clinician and an individual patient.3

    While RST have been examined for use in trauma surgery and in surgical critical care, few have been explicitly validated in the emergency general surgery (EGS) patient population.7 The American Association for the Surgery of Trauma (AAST) developed a comprehensive definition of EGS designed to encompass both operative and non-operative patients, using 621 International Classification of Disease-9th Revision diagnosis codes.8 These diagnoses encompassed 11 surgical areas ranging from general abdominal conditions to vascular conditions, cardiothoracic conditions and resuscitation. Given these differing etiologies and acute physiologic derangements, risk stratification in EGS presents a unique challenge. The ability to accurately predict individualized risk in this varied and vulnerable patient population is especially important at both the population level (probabilistic models) for auditing outcomes and quality of care evaluation as well as at the individual patient level (deterministic models) for safe triaging, treatment planning and shared decision making.

    In 2014, recognizing the urgency and unique challenges in risk prediction for the EGS population, the AAST Patient Assessment Committee established a grading system for the uniform reporting of anatomic severity of illness for 16 common EGS procedures.9–11 The goal of this body of work was to facilitate reliable scoring for risk stratification, outcome analysis, quality improvement and resource management.9 It is in this context that the AAST Patient Assessment Committee has undertaken the current review of existing RST to evaluate their applicability to EGS. We believe the ideal RST for EGS will meet six criteria for the generalized EGS patient population:

    1. Accurately quantify morbidity and mortality risk in the EGS population.

    2. Use readily obtainable objective data.

    3. Be applicable early prior to a surgical intervention.

    4. Be applicable in non-operative cases.

    5. Can be used for auditing purposes.

    6. Application that facilitates use in clinical practice.

    In the following review, we briefly discuss trauma and surgical critical care scoring systems, then focus on surgical RST with a close examination of the above characteristics. Our findings are summarized in table 1.

    View this table:
    Table 1

    Criteria fulfillment of each evaluated risk stratification tool

    Trauma scoring systems

    Trauma scoring systems can be categorized into three general types: anatomic, physiologic and combined scoring systems (figure 1). While they differ in specific focus, each category employs clinical data gathered at the time of initial survey to predict risk of morbidity and mortality following traumatic injury. Such information can further be employed for purposes of resource allocation, clinical triage and quality assurance. Anatomic scoring systems are based on anatomical classification of injury and typically do not include data reflective of physiologic status or comorbidities.6 12–36

    Figure 1
    Figure 1

    Trauma scoring systems.

    Trauma scoring systems allow for injury-focused, validated evaluations of complication and death risk. Scores are easily calculated using readily available objective data to make accurate predictions of mortality. Trauma scoring systems, however, are injury based and typically do not evaluate physiological parameters or comorbidities that may affect EGS patient outcome. Therefore, use of trauma scoring systems in this patient population is neither appropriate nor comprehensive.

    Critical care scoring systems

    There are many scoring systems used to evaluate mortality in the intensive care unit (ICU) patient. Generally based on data extracted at the time of ICU admission, and sometimes supplemented by further interval information, ICU-focused RST may be based on individual physiologic systems or more broadly encompassing. Prognostic ICU scores account for more than one organ system and evaluate the patient in a more global fashion (figure 2).37–65

    Figure 2
    Figure 2

    Critical care scoring systems.

    Generally, scoring systems designed for use in the critical care setting incorporate data points that reflect both prehospital health status and acute physiology. As comorbid conditions and acute health status are essential to the risk evaluation of the EGS patients, these scores may be useful in a subset of the EGS population. They are well validated in ICU populations and interval calculations allow for evaluation of disease progression. However, none of these scores has been explicitly validated for surgical patients in the perioperative setting, rendering them inappropriate for use in the EGS population outside the ICU.

    Surgical risk stratification tools

    Surgical RST, used to predict perioperative morbidity and mortality, fall under two general categories: physiologic scores and risk prediction models. These tools are explicitly designed for use in the perioperative setting. The applicability of some has expanded to the realm of quality evaluation and can form a scaffold for evidence based recommendations in practice guidelines.

    American Society of Anesthesiologists Physical Status Grading

    The American Society of Anesthesiologists Physical Status Grading (ASA-PS) was introduced in 1941 as a preoperative grading of the physical status of patients, and was subsequently modified in 1963.66 67 There are five grades based on the presence or absence of mild to serious life-threatening systemic disease, with an additional classification of E designated for emergency surgery. Higher ASA-PS grades have been correlated with prolonged postoperative in-hospital stay, postoperative surgical ICU admission and the development of serious postoperative sepsis, although studies have also demonstrated overestimates of mortality using this grading system.68–70

    ASA-PS has generally been shown to be an acceptable predictor of perioperative mortality and by design is applicable for use early in patient evaluation, can be incorporated into audits and facilities clinical practice. Its specific applicability in the EGS patient population has not been examined, and while the score is based on factors that may be available in a patients chart, there is some subjectivity to this scores tabulation, leading to interobserver variability.

    Criteria met: 3, 4, 5, 6

    Criteria not met: 1, 2

    ACS-NSQIP universal surgical risk calculator

    The American College of Surgeons National Surgical Quality Improvement Programme (ACS-NSQIP) database was used to develop a web-based tool to estimate the surgical risk of most operations. The resulting Universal Surgical Risk Calculator tool requires 21 preoperative factors (demographics, comorbidities, procedure), using regression models to predict mortality as well as six postoperative complications based on the preoperative risk factors.3 This surgical risk calculator has good discrimination and calibration and has only slight differences from previous procedure-specific ACS-NSQIP calculators. The ACS-NSQIP Universal Surgical Risk Calculator has been externally validated in an EGS population and was found to have a slight underestimation of the risk of EGS compared with the risk of elective surgery. While these differences were statistically significant, they were small with an observed to expected mortality ratio of 1.03 for EGS, leading the authors to conclude that it is applicable to emergency as well as elective patients.71 It is much more comprehensive than other risk calculators and has a web application which can be easily used.72

    Overall, the ACS-NSQIP Universal Surgical Risk Calculator meets our specified criteria. It is reasonably accurate in terms of estimation of postoperative death and complication in EGS, uses readily obtainable objective data, can be used in early phases of care and can be used for auditing purposes. Its calculation requires access to an internet-based portal, which generally is facilitated in clinical practice, but may limit its use in certain environments. One notable fact, however, is that the ACS-NSQIP Universal Surgical Risk Calculator requires a CPT code for a specific operation, and therefore is not applicable to non-operative EGS patients.

    Criteria met: 1, 2, 3, 5, 6

    Criteria not met: 4

    Charlson Comorbidity Index

    The Charlson Comorbidity Index (CCI) was originally developed to facilitate classification of risk of death extrapolated from comorbidities, to be used in longitudinal studies.73 It was developed using 1 year mortality data in patients admitted to the medical service at New York Hospital and validated in patients with primary breast cancer at Yale New Haven Hospital over a 10-year follow-up period. Nineteen comorbid conditions were classified and a weighted index was developed that accounted for the number and seriousness of comorbid diseases.74 This was later modified by Charlson to add age to the comorbidity index, formulating the Charlson Age-Comorbidity Index (CACI). The CACI combines 19 medical conditions weighted 1–6, with age weighted 1 for every decade past 40 years.74

    The CACI has been validated in the setting of medical patients, critically ill and trauma patients as well as many elective surgical settings. Evidence to evaluate the role of CCI and CACI in the emergency setting is sparse. A large review of Saudi Arabian patients demonstrated CCI as a reliable comorbidity index for prediction outcomes in acute non-traumatic surgical patients.75 An Australian study of 2819 patients demonstrated a correlation between the CCI as a measure of preoperative comorbidity and postoperative mortality in trauma patients.76 One study of 257 EGS patients undergoing an operation found that increasing CACI scores were associated with higher rates of 30-day postoperative mortality.77

    Despite a lack of incorporation of explicitly surgical factors, the literature suggests that CCI and CACI may accurately estimate morbidity and mortality in the EGS population. These scores are based on a broad range of comorbid conditions and objective data points, can be applied early and can be used for auditing. The limitation to the use of these scores in EGS falls in their sometimes cumbersome calculation; with so many factors to account for, they are not well suited to the time sensitive preoperative environment.

    Criteria met: 1, 2, 3, 4, 5

    Criteria not met: 6

    Emergency Surgery Acuity Score

    The Emergency Surgery Acuity Score is a preoperative risk stratification system that predicts perioperative mortality in emergency surgery patients.7 It captures both patient comorbidities and the acute physiology at presentation. This score was formed by multiple logistic regression analyses of independent predictors of mortality based on preoperative variables in ACS-NSQIP data fields. The ESAS includes 3 demographic variables, 10 comorbidities and 9 laboratory variables. Based on the relative impact of these 22 predictors, using weighted averages, a score is derived that ranges from 0 to 29 points. The observed probability of 30-day mortality is 0% at a score of 0, 36% at a score of 11, 75% at a score of 19% and 100% for scores of 22 or greater. The ESAS has been validated using the 2012 ACS-NSQIP database and has demonstrated a higher discriminatory power in comparison with ASA-PS.

    ESAS has demonstrated the ability to differentiate mortality accurately, although our understanding of its use may benefit from prospective examination. Further work is needed to understand the applicability of this score to EGS postoperative morbidity. It is based entirely on objective factors that can be easily identified preoperatively. The score could be standardized to be used for auditing and its straightforward calculation model renders it easy to apply clinically, although no ready-to-use application exists as of yet. While this risk score could be used in non-operative patients as it does not require any surgical variables, it was derived from NSQIP data which only contains operative cases. Therefore, as it was derived and validated using only operative patients, further validation will be necessary for this score to be reliably applied to non-operative EGS patients.

    Criteria met: 1, 2, 3, 5

    Criteria not met: 4, 6

    Physiological Emergency Surgery Acuity Score

    Like ESAS, the Physiological Emergency Surgery Acuity Score (PESAS), was derived from sequential analyses of ACS-NSQIP data, this time focused on information collected in 2011. The resulting score is composed of 10 physiologic points linked to advanced clinical acuity and postoperative mortality.78 PESAS was subsequently internally validated on ACS-NSQIP data from 2012, yielding a c-statistic of 0.79 (95% CI receiver operating characteristic 0.7801 to 0.8025).79

    This score was explicitly created for use in the EGS patient population and is designed to assess operative risk of mortality using objective data in a time-sensitive, applicable manner. However, this score may not fully capture the extent of a patient’s coexisting conditions and is not validated to assess morbidity. This score can be applied early in a patients care trajectory, can be used for auditing purposes and is appropriate for clinical use. As discussed with ESAS, as this score was derived and validated using operative patients, further validation is needed for reliable use with non-operative EGS patients.

    Criteria met: 1, 2, 3, 5

    Criteria not met: 4, 6

    Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity

    The Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity (POSSUM) was devised in 1996 in the UK as a simple scoring system that could be used in surgical audit.80 The original POSSUM is a 12 factor, four grade, physiological score (ranging from 12 through 88) plus a 6-factor operative severity score (ranging from 6 through 44). These two scores are inserted into two formulas to give predicted morbidity and mortality. Correlation in predicted versus observed rates of mortality and morbidity has been demonstrated.80 However, the POSSUM overestimates mortality rates, especially in low-risk procedures. The lowest possible scores that POSSUM allows are 12 for physiology and 6 for operative severity, which predicts the lowest risk of death as 1.08%.80

    When Whitely et al 81 applied the POSSUM in Portsmouth, UK hospitals, it was found that the POSSUM overpredicted the total number of deaths by a factor of 2, performing worst in low-risk patients (by a factor of 6 in those with predicted risk of death of 10% or less). Using these data, their team remodeled the tool, creating a Portsmouth predictor equation for mortality. This Portsmouth POSSUM (P-POSSUM) still overpredicts death in the low-risk patient with the lowest predictive risk for death of 0.20%, which is still higher than expected for patients with no physiologic abnormalities and age <61 years.81 More recently, web-based calculators have become available, making this scoring system more user-friendly.

    The limited data on use of POSSUM scores in EGS patients have demonstrated overestimates of mortality rates for low-risk procedures. As this score is based partially on operative data, it cannot be used in the preoperative setting and will not be useful for non-operative cases. Some data points, such as chest radiograph interpretation, are subjective in nature. This may limit its use for auditing purposes and its generalized use in modern EGS practice.

    Criteria met: 5, 6

    Criteria not met: 1, 2, 3, 4

    Perioperative Mortality Risk Score

    The Perioperative Mortality Risk Score (PMRS) is a risk score for 30-day postoperative mortality derived from a study of surgical patients aged 70 years or older from three large metropolitan teaching hospitals.82 Risk factors for 30-day mortality were determined by multivariate analysis and the risk score for each of six factors were summed to give an overall PMRS. The score was initially validated using 256 patients from a regional hospital.

    While appropriate for the elderly EGS subset, this score is not generalizable across the entirety of the EGS population. It is based on readily available preoperative and postoperative data, although obviously half of these factors cannot be evaluated in the preoperative period. The lack of overall applicability of this score may limit its utility in audits. The calculation is not complex and can be applied at the bedside.

    Criteria met: 2, 6

    Criteria not met: 1, 3, 4, 5

    Surgical Apgar Score

    The Surgical Apgar Score (SAS) is a 10-point score based on the lowest heart rate, lowest mean arterial pressure and estimated blood loss intraoperatively.1 It has been validated to discriminate well between groups of patients at high and low risk of major complications and death within 30 days of operation after colectomy and general/vascular surgery. However, original studies of this score included only 3%–6% emergency operations. Further study has suggested that while the SAS has strong predictive value for postoperative morbidity, it is weakly discriminative and thus should not be used as a standalone tool for evaluation.66 Because the variables are influenced by the performance of the medical teams and the patient’s earlier conditions, SAS does not allow comparison of quality between institutions or practitioners.

    The SAS direct applicability to risk evaluation in EGS has not been adequately studied. It is an operatively focused score based on attained objective data points and therefore cannot be used in the preoperative setting. It is not appropriate for non-operative EGS patients. It can be readily applied following completion of the surgical procedure. However, it is not appropriate for auditing purposes as it reflects intraoperative care, not solely patient risk.

    Criteria met: 2, 6

    Criteria not met: 1, 3, 4, 5

    Surgical Mortality Score

    The original Surgical Mortality Score (SMS) is a risk stratification model for in-hospital mortality for patients undergoing surgical procedures developed across a range of surgical specialties in England.83 The model uses data from an existing administrative database: the hospital Patient Administration System at a London university hospital. SMS risk factors for in-hospital mortality include surgical specialty, age at operation, gender, mode of surgery (emergency or elective), time of onset of surgical procedure and median operating time. The SMS model is used to calculate the mortality risk ratio for each specialty, giving an SMS predicted in-hospital mortality that can be compared with the observed in-hospital mortality. The exact binomial 95% CI for the observed in-hospital mortality rate is then divided by the SMS-predicted risk to obtain the 95% CI of the mortality risk ratio.

    This score has not been explicitly studied in the EGS population; its ability to accurately estimate morbidity and mortality for EGS is unknown. It is based on available data points, a portion of which are extracted from operating room chart review, which limits preoperative applicability and relevance for auditing. Its calculation is not entirely straightforward, which can render it cumbersome for generalized practice.

    Criteria met: 2

    Criteria not met: 1, 3, 4, 5, 6

    Surgical Outcome Risk Tool

    The Surgical Outcome Risk Tool (SORT) is a preoperative risk prediction tool for death within 30 days of surgery in adults undergoing non-cardiac, non-neurologic inpatient surgery.84 It was developed in 2014 in the UK. The six variables included are ASA-PS grade (III, IV and V), urgency of surgery (expedited, urgent, immediate), specialty, severity of surgery, presence of cancer and age. Model coefficients were determined and used to develop a formula for a risk score. Internal validation in the UK demonstrated the SORT to be more accurate than the ASA-PS and the SRS. A web-based calculator is available as well as an app to aid in calculations of the SORT.85

    Although it does account for emergent case status, SORT tool has not been explicitly studied in the EGS patient population, thus its capacity to accurately estimate morbidity and mortality risk in this patient subset is unknown. Its inclusion of ASA scoring renders its calculation susceptible to some degree of subjectivity. This score is explicitly designed for use in the preoperative setting and generally can be used for auditing. Calculation of this score is relatively straightforward and, if appropriately validated, could be well suited for use in clinical practice.

    Criteria met: 3, 5, 6

    Criteria not met: 1, 2, 4

    Surgical Risk Scale

    The Surgical Risk Scale85 was developed as an alternative system for comparative audit that used easily collected clinical data independent of the surgeon. The Surgical Risk Scale has three components: the Confidential Enquiry into Perioperative Deaths grade (categorized as elective, scheduled, urgent and emergency), the ASA-PS grade and the British United Provident Association (BUPA) operative grade (categorized as major, intermediate, major, major plus and complex major).

    This tool has not been explicitly studied for risk prediction in EGS and the Surgical Risk Scale has only been validated to predict death in single-center studies in the UK. The scoring is somewhat subjective, through incorporation of ASA status and BUPA categorization. Its use requires some familiarity with BUPA categories, which may not be readily available. This score can be applied early in a patients care course but may not be ideal for auditing given possible variances in interpretation of ASA and BUPA scoring. It also obviously cannot be used in non-operative patients. Although not a complex set of calculations, subjectivity and lack of familiarity with BUPA may limit the clinical relevance of this score.

    Criteria met: 3

    Criteria not met: 1, 2, 4, 5, 6

    Surgical Risk Score

    The Surgical Risk Score is an ASA-PS-based model to predict mortality based on preoperative data alone.86 It was formulated using data of all types of operations excluding cardiac surgeries and caesarian section. This score includes ASA-PS status, age, type of surgery (elective, urgent emergency) and degree of surgery (minor, moderate or major as described by the modified Johns Hopkins surgical severity score).

    Like the Surgical Risk Scale, this tool has not been explicitly studied for risk prediction in EGS. Its incorporation of ASA status and Johns Hopkins Surgical Severity Score introduces a degree of subjectivity. The score can be applied early in a patients care and may be clinically helpful, but more work would be needed to describe its applicability to this patient subset. It also cannot be used in non-operative patients.

    Criteria met: 3

    Criteria not met: 1, 2, 4, 5, 6

    Track and trigger scoring systems

    Track and trigger systems are defined as scoring systems that rely on periodic or interval measurements of data points (afferent information) with predetermined action plans (efferent action) when certain data or scoring thresholds are reached.87 Such systems are geared towards early identification of, and subsequent team-based response to, changes in patient physiologic status linked to adverse in-hospital outcomes.87–89 Scores can be single parameter-based, focused on a single clinical data point, multiple parameter-based or derived from weighted aggregate models90 (figure 3). Single-parameter systems require that only one trigger threshold be met before a response is issued while multiparameter systems are triggered on combinations of physiologic data points.78 90–95 Aggregated weighted systems are derived from comparatively complex, statistically weighted calculations that allow for evaluation of multiple variables in the context of degree of abnormality.96–100 Notable examples include the Modified Early Warning System (MEWS), the Worthing Physiologic Scoring System (WPSS), the Rothman Index (RI) and the Acute Laboratory Risk of Mortality Score (ALaRMS).96–100

    Figure 3
    Figure 3

    Track and trigger scoring systems.

    The MEWS uses statistical calibration to weigh the data points by which the Early Warning System (EWS) is defined, with the intent of adjusting the basic EWS to reflect the needs of a broad variety of patient populations.96 Scoring has been positively associated with risk of death and ICU admission.96 Peris et al investigated the effect of implementing the MEWS in emergency surgical patients as a means to triage patients to the ICU or to a non-ICU level of care, finding no significant difference in in-hospital mortality rates.97 The WPSS, also an EWS derivative, was built from data collected from emergency department admissions. When compared with the EWS, WPSS has been shown to exhibit improved discrimination in patients at risk for negative outcomes.98

    The RI was developed to be a general measure of individual patient condition using 26 clinical variables commonly available in the electronic medical record (EMR) including vital signs, lab results, cardiac rhythms and nursing assessments. This scoring index was aimed to supply real time warnings that predicted poor outcomes and identified them early enough that interventions could be made to abort those outcomes.99 This score is reliant on the EMR to synthesize and calculate scores on a rolling basis. Each time one of the included variables is updated in the EMR, a new RI is calculated. The score intentionally avoids demographic data that are static and focuses on readily available dynamic variables associated with patient condition. This is to allow for alterations in the patient clinical trajectory to emerge. The RI was shown to be an accurate predictor of 24 hours mortality.99 The RI has been compared with APACHE III (to predict outcome prior to transfer to the ICU) and MEWS score (to identify clinical deterioration), and the RI has shown favorable correlation.

    The ALaRMS is a scoring system composed of 23 laboratory values that can be readily sampled from the EMR in an automated fashion. It is based on previous studies that show that laboratory data can predict outcomes in both disease-specific as well as generic patient populations. In evaluating over 2 million discharges from the electronic health record, using numeric laboratory results combined with comorbidities as evaluated by diagnostic codes, the ALaRMS score has demonstrated excellent predictive ability in high volume admissions, in both medical and surgical patient populations, and is deemed the strongest of the track and trigger systems.100

    Track and trigger systems, especially those that are aggregated and weighted, have great potential to accurately predict risk in individual patients, although precision of risk prediction in EGS patients is still unknown. These systems can be automated, use readily available data and can be continuously updated in real time throughout a patients care. Application to non-operative patients may be feasible and these scores tend to be clinically facile.

    Criteria met: 2, 3, 4, 5, 6

    Criteria not met: 1

    Conclusion

    This review has explored the wide variety of RST available with potential usage in the EGS population. While several scoring systems and models have shown promise for use in intensive care and in trauma surgery, few have been explicitly studied for evaluation of the EGS patient. From a patient-specific standpoint, the past decade has revealed that this population is unique in that its propensity towards pre-existing comorbid conditions paired with acute pathophysiology renders it exceptionally susceptible to postoperative morbidity and mortality.101 As our understanding of the unique risks of EGS evolves, RST must also evolve to account for these risks. In this vulnerable population, evidence-based RST that accurately predict individual risk will facilitate decision making, enhance informed consent and permit the comparison of outcomes in patient groups that share similar comorbidity profiles. Such an RST must incorporate information regarding an individual patients acute and chronic physiology through readily accessible data points, and expeditious use must be feasible.

    While valuable in the setting of acute injury, trauma scoring systems do not comprehensively evaluate the comorbid conditions inherent to the EGS population and are therefore not ideal for use in EGS.6 19 22 27 The CCI is reflective of comorbid conditions but does not necessarily capture data points reflective of acute physiologic changes that may contribute to postoperative outcomes; additionally, it does not assess risk from a procedural standpoint.73 77 SRS, POSSUM and P-POSSUM contain comprehensive information but have shown significant limitations in EGS.7 80 85 Track and trigger tools have demonstrated that they may have a role to play in the care of the EGS patient on the floor or in the ICU, but their role in the setting of an operation is not presently clear.

    ESAS, developed for EGS from ACS-NSQIP data, incorporates information reflective both of a patient’s underlying comorbidities and of acute status. PESAS has been designed for assessment of perioperative acuity and mortality risk assessment. ESAS meets most of our definition of an ideal scoring system. It accurately predicts mortality in the EGS population, it uses readily obtainable objective data, it can be used early in the course of an illness and could be used for auditing purposes. However, it does not yet predict complications, it is not applicable to non-operative patients, and is not yet available as a consumer application. The ACS-NSQIP surgical risk calculator incorporates a comprehensive body of preoperative data and accounts for the specific planned procedure. It has been previously validated in the EGS population and has shown broad statistical reliability.71 102 103 It is the best fit of our definition of the ideal scoring tool at this time, as it accurately quantifies both morbidity and mortality in the EGS population, uses readily obtainable objective data, is applicable early in the course of disease, can be used for auditing purposes and has an easy-to-use consumer application. However, it is also not applicable to non-operative patients.

    Given the limitations of other RST and the comprehensive incorporation of data offered by the NSQIP surgical risk calculator, at present, our group recommends the use of NSQIP for use in the EGS population. ESAS also shows promise in this population. Further tailoring to non-operative EGS patients, addition of risk prediction for complications and development of a consumer application may bring ESAS closer to being the precise, population-specific, scoring system that this patient population needs.

    References

    1. ↵
      2. Gawande AA ,
      3. Kwaan MR ,
      4. Regenbogen SE ,
      5. Lipsitz SA ,
      6. Zinner MJ
      . An Apgar score for surgery. J Am Coll Surg 2007;204:201–8.doi:10.1016/j.jamcollsurg.2006.11.011
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      2. Moonesinghe SR ,
      3. Mythen MG ,
      4. Das P ,
      5. Rowan KM ,
      6. Grocott MP
      . Risk stratification tools for predicting morbidity and mortality in adult patients undergoing major surgery: qualitative systematic review. Anesthesiology 2013;119:959–81.doi:10.1097/ALN.0b013e3182a4e94d
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      2. Bilimoria KY ,
      3. Liu Y ,
      4. Paruch JL ,
      5. Zhou L ,
      6. Kmiecik TE ,
      7. Ko CY ,
      8. Cohen ME
      . Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg 2013;217:833–42.doi:10.1016/j.jamcollsurg.2013.07.385
      OpenUrlCrossRefPubMed
    4. ↵
      2. Adams ST ,
      3. Leveson SH
      . Clinical prediction rules. BMJ 2012;344:d8312.doi:10.1136/bmj.d8312
      OpenUrlFREE Full Text
    5. ↵
      2. Chand M ,
      3. Armstrong T ,
      4. Britton G ,
      5. Nash GF
      . How and why do we measure surgical risk? J R Soc Med 2007;100:508–12.doi:10.1177/014107680710001113
      OpenUrlCrossRefPubMed
    6. ↵
      2. Baker SP ,
      3. O’Neill B ,
      4. Haddon W ,
      5. Long WB
      . The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma 1974;14:187–96.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      2. Sangji NF ,
      3. Bohnen JD ,
      4. Ramly EP ,
      5. Yeh DD ,
      6. King DR ,
      7. DeMoya M ,
      8. Butler K ,
      9. Fagenholz PJ ,
      10. Velmahos GC ,
      11. Chang DC , et al
      . Derivation and validation of a novel Emergency Surgery Acuity Score (ESAS). J Trauma Acute Care Surg 2016;81:213–20.doi:10.1097/TA.0000000000001059
      OpenUrl
    8. ↵
      2. Shafi S ,
      3. Aboutanos MB ,
      4. Agarwal S ,
      5. Brown CV ,
      6. Crandall M ,
      7. Feliciano DV ,
      8. Guillamondegui O ,
      9. Haider A ,
      10. Inaba K ,
      11. Osler TM , et al
      . Emergency general surgery: definition and estimated burden of disease. J Trauma Acute Care Surg 2013;74:1092–7.doi:10.1097/TA.0b013e31827e1bc7
      OpenUrlCrossRefPubMed
    9. ↵
      2. Tominaga GT ,
      3. Staudenmayer KL ,
      4. Shafi S ,
      5. Schuster KM ,
      6. Savage SA ,
      7. Ross S ,
      8. Muskat P ,
      9. Mowery NT ,
      10. Miller P ,
      11. Inaba K , et al
      . The American Association for the Surgery of Trauma grading scale for 16 emergency general surgery conditions: Disease-specific criteria characterizing anatomic severity grading. J Trauma Acute Care Surg 2016;81:593–602.doi:10.1097/TA.0000000000001127
      OpenUrl
    10. ↵
      2. Shafi S ,
      3. Aboutanos M ,
      4. Brown CV ,
      5. Ciesla D ,
      6. Cohen MJ ,
      7. Crandall ML ,
      8. Inaba K ,
      9. Miller PR ,
      10. Mowery NT
      . American Association for the Surgery of Trauma Committee on Patient Assessment and Outcomes. Measuring anatomic severity of disease in emergency general surgery. J Trauma Acute Care Surg 2014;76:884–7.doi:10.1097/TA.0b013e3182aafdba
      OpenUrlPubMed
    11. ↵
      2. Crandall ML ,
      3. Agarwal S ,
      4. Muskat P ,
      5. Ross S ,
      6. Savage S ,
      7. Schuster K ,
      8. Tominaga GT ,
      9. Shafi S
      . American Association for the Surgery of TraumaCommittee on Patient Assessment and Outcomes. Application of a uniform anatomic grading system to measure disease severity in eight emergency general surgical illnesses. J Trauma Acute Care Surg 2014;77:705–8.doi:10.1097/TA.0000000000000444
      OpenUrl
    12. ↵
      2. States JD
      . The abbreviated and the comprehensive research injury scales. Society of Automotive Engineers I. New York: STAPP Car Crash Journal, 1969:S282–94.
    13. ↵
      American Association for Automotive Medicine. Abbreviated injury scale: University of Chicago, 1985.
    14. ↵
      2. Clark DE ,
      3. Winchell RJ
      . Risk adjustment for injured patients using administrative data. J Trauma 2004;57:130–40.doi:10.1097/01.TA.0000114240.58834.84
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      2. OKeefe JGJ . et al.
      Measurement of injury severity and co-morbidity. In: Rivera FP CP , Koepsell TD , Grossman DC , Maier RV , . eds. Injury control: Cambridge Univserity Press, 2001.
    16. ↵
      Expert group in Injury Severity. Discussion document on injury severity measurement in administrative data sets.
    17. ↵
      2. MacKenzie EJ ,
      3. Shapiro S ,
      4. Moody M ,
      5. Siegel JH ,
      6. Smith RT
      . Predicting posttrauma functional disability for individuals without severe brain injury. Med Care 1986;24:377–87.doi:10.1097/00005650-198605000-00001
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      2. MacKenzie EJ ,
      3. Shapiro S ,
      4. Eastham JN
      . The Abbreviated Injury Scale and Injury Severity Score. Levels of inter- and intrarater reliability. Med Care 1985;23:823–35.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      2. Osler T ,
      3. Baker SP ,
      4. Long W
      . A modification of the injury severity score that both improves accuracy and simplifies scoring. J Trauma 1997;43:922–6.doi:10.1097/00005373-199712000-00009
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      2. de Munter L ,
      3. Polinder S ,
      4. Lansink KW ,
      5. Cnossen MC ,
      6. Steyerberg EW ,
      7. de Jongh MA
      . Mortality prediction models in the general trauma population: A systematic review. Injury 2017;48:221–9.doi:10.1016/j.injury.2016.12.009
      OpenUrl
    21. ↵
      2. Oreskovich MR ,
      3. Howard JD ,
      4. Copass MK ,
      5. Carrico CJ
      . Geriatric trauma: injury patterns and outcome. J Trauma 1984;24:565–72.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      2. Champion HR ,
      3. Sacco WJ ,
      4. Carnazzo AJ ,
      5. Copes W ,
      6. Fouty WJ
      . Trauma score. Crit Care Med 1981;9:672–6.doi:10.1097/00003246-198109000-00015
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      2. Moreau M ,
      3. Gainer PS ,
      4. Champion H ,
      5. Sacco WJ
      . Application of the trauma score in the prehospital setting. Ann Emerg Med 1985;14:1049–54.doi:10.1016/S0196-0644(85)80917-3
      OpenUrlCrossRefPubMed
    24. ↵
      2. Sacco WJ ,
      3. Champion H ,
      4. Stega M
      . Trauma care evaluation: Baltimore University Park Press, 1984.
    25. ↵
      2. Sacco WJ ,
      3. Champion HR ,
      4. Gainer PS ,
      5. Morelli SA ,
      6. Fallen S ,
      7. Lawnick MA
      . The Trauma Score as applied to penetrating trauma. Ann Emerg Med 1984;13:415–8.doi:10.1016/S0196-0644(84)80002-5
      OpenUrlPubMed
    26. ↵
      2. Champion HR ,
      3. Sacco WJ ,
      4. Hunt TK
      . Trauma severity scoring to predict mortality. World J Surg 1983;7:4–11.doi:10.1007/BF01655906
      OpenUrlCrossRefPubMed
    27. ↵
      2. Champion HR ,
      3. Sacco WJ ,
      4. Copes WS ,
      5. Gann DS ,
      6. Gennarelli TA ,
      7. Flanagan ME
      . A revision of the Trauma Score. J Trauma 1989;29:623–9.doi:10.1097/00005373-198905000-00017
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      2. Moore L ,
      3. Lavoie A ,
      4. Abdous B ,
      5. Le Sage N ,
      6. Liberman M ,
      7. Bergeron E ,
      8. Emond M
      . Unification of the revised trauma score. J Trauma 2006;61:718–22.doi:10.1097/01.ta.0000197906.28846.87
      OpenUrlPubMedWeb of Science
    29. ↵
      2. Gabbe BJ ,
      3. Cameron PA ,
      4. Finch CF
      . Is the revised trauma score still useful? ANZ J Surg 2003;73:944–8.doi:10.1046/j.1445-1433.2003.02833.x
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      2. Champion HR ,
      3. Copes WS ,
      4. Sacco WJ ,
      5. Lawnick MM ,
      6. Keast SL ,
      7. Bain LW ,
      8. Flanagan ME ,
      9. Frey CF
      . The Major Trauma Outcome Study: establishing national norms for trauma care. J Trauma 1990;30:1356–65.
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      2. Boyd CR ,
      3. Tolson MA ,
      4. Copes WS
      . Evaluating trauma care: the TRISS method. Trauma Score and the Injury Severity Score. J Trauma 1987;27:370–8.
      OpenUrlCrossRefPubMedWeb of Science
    32. ↵
      2. Singh J ,
      3. Gupta G ,
      4. Garg R ,
      5. Gupta A
      . Evaluation of trauma and prediction of outcome using TRISS method. J Emerg Trauma Shock 2011;4:446–9.doi:10.4103/0974-2700.86626
      OpenUrlPubMed
    33. ↵
      2. Chico-Fernández M ,
      3. Llompart-Pou JA ,
      4. Sánchez-Casado M ,
      5. Alberdi-Odriozola F ,
      6. Guerrero-López F ,
      7. Mayor-García MD ,
      8. Egea-Guerrero JJ ,
      9. Fernández-Ortega JF ,
      10. Bueno-González A ,
      11. González-Robledo J , et al
      . Mortality prediction using TRISS methodology in the Spanish ICU Trauma Registry (RETRAUCI). Med Intensiva 2016;40:395–402.doi:10.1016/j.medin.2015.11.003
      OpenUrl
    34. ↵
      2. Cayten CG ,
      3. Stahl WM ,
      4. Murphy JG ,
      5. Agarwal N ,
      6. Byrne DW
      . Limitations of the TRISS method for interhospital comparisons: a multihospital study. J Trauma 1991;31:471–81.
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      2. Gabbe BJ ,
      3. Cameron PA ,
      4. Wolfe R
      . TRISS: does it get better than this? Acad Emerg Med 2004;11:181–6.doi:10.1111/j.1553-2712.2004.tb01432.x
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      2. Domingues CA ,
      3. Nogueira LS ,
      4. Settervall CH ,
      5. Sousa RM
      . Performance of Trauma and Injury Severity Score(TRISS) adjustments: an integrative review. Rev Esc Enferm USP 2015;49 Spec No:138–46.doi:10.1590/S0080-623420150000700020
      OpenUrl
    37. ↵
      2. Knaus WA ,
      3. Draper EA ,
      4. Wagner DP ,
      5. Zimmerman JE
      . APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818–29.
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      2. Knaus WA ,
      3. Zimmerman JE ,
      4. Wagner DP ,
      5. Draper EA ,
      6. Lawrence DE
      . APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 1981;9:591–7.doi:10.1097/00003246-198108000-00008
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      2. Bouch DC ,
      3. Thompson JP
      . Severity scoring systems in the critically ill. Continuing Education in Anaesthesia Critical Care & Pain 2008;8:181–5.doi:10.1093/bjaceaccp/mkn033
      OpenUrlCrossRef
    40. ↵
      2. Capuzzo M ,
      3. Valpondi V ,
      4. Sgarbi A ,
      5. Bortolazzi S ,
      6. Pavoni V ,
      7. Gilli G ,
      8. Candini G ,
      9. Gritti G ,
      10. Alvisi R
      . Validation of severity scoring systems SAPS II and APACHE II in a single-center population. Intensive Care Med 2000;26:1779–85.doi:10.1007/s001340000715
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      2. Headley J ,
      3. Theriault R ,
      4. Smith TL
      . Independent validation of APACHE II severity of illness score for predicting mortality in patients with breast cancer admitted to the intensive care unit. Cancer 1992;70:497–503.doi:10.1002/1097-0142(19920715)70:2<497::AID-CNCR2820700220>3.0.CO;2-H
      OpenUrlPubMed
    42. ↵
      2. Sánchez-Hurtado LA ,
      3. Ángeles-Veléz A ,
      4. Tejeda-Huezo BC ,
      5. García-Cruz JC ,
      6. Juárez-Cedillo T
      . Validation of a prognostic score for mortality in elderly patients admitted to Intensive Care Unit. Indian J Crit Care Med 2016;20:695–700.doi:10.4103/0972-5229.195702
      OpenUrl
    43. ↵
      2. Beck DH ,
      3. Taylor BL ,
      4. Millar B ,
      5. Smith GB
      . Prediction of outcome from intensive care: a prospective cohort study comparing Acute Physiology and Chronic Health Evaluation II and III prognostic systems in a United Kingdom intensive care unit. Crit Care Med 1997;25:9–15.
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      2. Moseson EM ,
      3. Zhuo H ,
      4. Chu J ,
      5. Stein JC ,
      6. Matthay MA ,
      7. Kangelaris KN ,
      8. Liu KD ,
      9. Calfee CS
      . Intensive care unit scoring systems outperform emergency department scoring systems for mortality prediction in critically ill patients: a prospective cohort study. J Intensive Care 2014;2:40.doi:10.1186/2052-0492-2-40
      OpenUrl
    45. ↵
      2. Le Gall JR ,
      3. Loirat P ,
      4. Alperovitch A ,
      5. Glaser P ,
      6. Granthil C ,
      7. Mathieu D ,
      8. Mercier P ,
      9. Thomas R ,
      10. Villers D
      . A simplified acute physiology score for ICU patients. Crit Care Med 1984;12:975–7.doi:10.1097/00003246-198411000-00012
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      2. Le Gall JR ,
      3. Lemeshow S ,
      4. Saulnier F
      . A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 1993;270:2957–63.doi:10.1001/jama.1993.03510240069035
      OpenUrlCrossRefPubMedWeb of Science
    47. ↵
      2. Metnitz PG ,
      3. Moreno RP ,
      4. Almeida E ,
      5. Jordan B ,
      6. Bauer P ,
      7. Campos RA ,
      8. Iapichino G ,
      9. Edbrooke D ,
      10. Capuzzo M ,
      11. Le Gall JR
      . SAPS 3 Investigators. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med 2005;31:1336–44.doi:10.1007/s00134-005-2762-6
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      2. Keegan MT ,
      3. Gajic O ,
      4. Afessa B
      . Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance. Chest 2012;142:851–8.doi:10.1378/chest.11-2164
      OpenUrlCrossRefPubMed
    49. ↵
      2. Marshall JC ,
      3. Cook DJ ,
      4. Christou NV ,
      5. Bernard GR ,
      6. Sprung CL ,
      7. Sibbald WJ
      . Multiple organ dysfunction score: a reliable descriptor of a complex clinical outcome. Crit Care Med 1995;23:1638–52.
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      2. Le Gall JR ,
      3. Klar J ,
      4. Lemeshow S ,
      5. Saulnier F ,
      6. Alberti C ,
      7. Artigas A ,
      8. Teres D
      . The Logistic Organ Dysfunction system. A new way to assess organ dysfunction in the intensive care unit. ICU Scoring Group. JAMA 1996;276:802–10.
      OpenUrlCrossRefPubMedWeb of Science
    51. ↵
      2. Vincent JL ,
      3. Moreno R ,
      4. Takala J ,
      5. Willatts S ,
      6. De Mendonça A ,
      7. Bruining H ,
      8. Reinhart CK ,
      9. Suter PM ,
      10. Thijs LG
      . The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996;22:707–10.
      OpenUrlCrossRefPubMedWeb of Science
    52. ↵
      2. Metnitz PG ,
      3. Lang T ,
      4. Valentin A ,
      5. Steltzer H ,
      6. Krenn CG ,
      7. Le Gall JR
      . Evaluation of the logistic organ dysfunction system for the assessment of organ dysfunction and mortality in critically ill patients. Intensive Care Med 2001;27:992–8.doi:10.1007/s001340100888
      OpenUrlCrossRefPubMedWeb of Science
    53. ↵
      2. Kim TK ,
      3. Yoon JR
      . Comparison of the predictive power of the LODS and APACHE II scoring systems in a neurological intensive care unit. J Int Med Res 2012;40:777–86.doi:10.1177/147323001204000244
      OpenUrlPubMed
    54. ↵
      2. Padilha KG ,
      3. Cardose de Silva RM ,
      4. de Silva MCM
      . Patients organ dysfunction in the intensive care unit according to the logistic organ dysfunction system. Rev Esc Enferm USP 2009;43.
    55. ↵
      2. Namendys-Silva SA ,
      3. Texcocano-Becerra J ,
      4. Herrera-Gómez A
      . Application of the Sequential Organ Failure Assessment (SOFA) score to patients with cancer admitted to the intensive care unit. Am J Hosp Palliat Care 2009;26:341–6.doi:10.1177/1049909109333041
      OpenUrlCrossRefPubMed
    56. ↵
      2. Badreldin AM ,
      3. Doerr F ,
      4. Ismail MM ,
      5. Heldwein MB ,
      6. Lehmann T ,
      7. Bayer O ,
      8. Doenst T ,
      9. Hekmat K
      . Comparison between Sequential Organ Failure Assessment score (SOFA) and Cardiac Surgery Score (CASUS) for mortality prediction after cardiac surgery. Thorac Cardiovasc Surg 2012;60:035–42.doi:10.1055/s-0030-1270943
      OpenUrl
    57. ↵
      2. Craig DG ,
      3. Reid TW ,
      4. Wright EC ,
      5. Martin KG ,
      6. Davidson JS ,
      7. Hayes PC ,
      8. Simpson KJ
      . The sequential organ failure assessment (SOFA) score is prognostically superior to the model for end-stage liver disease (MELD) and MELD variants following paracetamol (acetaminophen) overdose. Aliment Pharmacol Ther 2012;35:705–13.doi:10.1111/j.1365-2036.2012.04996.x
      OpenUrlPubMed
    58. ↵
      2. Seymour CW ,
      3. Liu VX ,
      4. Iwashyna TJ ,
      5. Brunkhorst FM ,
      6. Rea TD ,
      7. Scherag A ,
      8. Rubenfeld G ,
      9. Kahn JM ,
      10. Shankar-Hari M ,
      11. Singer M , et al
      . Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016;315:762–74.doi:10.1001/jama.2016.0288
      OpenUrlCrossRefPubMed
    59. ↵
      2. Freund Y ,
      3. Lemachatti N ,
      4. Krastinova E ,
      5. Van Laer M ,
      6. Claessens YE ,
      7. Avondo A ,
      8. Occelli C ,
      9. Feral-Pierssens AL ,
      10. Truchot J ,
      11. Ortega M , et al
      . Prognostic accuracy of sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department. JAMA 2017;317:301–8.doi:10.1001/jama.2016.20329
      OpenUrl
    60. ↵
      2. Raith EP ,
      3. Udy AA ,
      4. Bailey M ,
      5. McGloughlin S ,
      6. MacIsaac C ,
      7. Bellomo R ,
      8. Pilcher DV
      . Australian and New Zealand Intensive Care Society (ANZICS) Centre for Outcomes and Resource Evaluation (CORE). Prognostic Accuracy of the SOFA Score, SIRS Criteria, and qSOFA Score for In-Hospital Mortality Among Adults With Suspected Infection Admitted to the Intensive Care Unit. JAMA 2017;317:290–300.doi:10.1001/jama.2016.20328
      OpenUrlCrossRef
    61. ↵
      2. Peres Bota D ,
      3. Melot C ,
      4. Lopes Ferreira F ,
      5. Nguyen Ba V ,
      6. Vincent JL
      . The Multiple Organ Dysfunction Score (MODS) versus the Sequential Organ Failure Assessment (SOFA) score in outcome prediction. Intensive Care Med 2002;28:1619–24.doi:10.1007/s00134-002-1491-3
      OpenUrlCrossRefPubMedWeb of Science
    62. ↵
      2. Khwannimit B
      . A comparison of three organ dysfunction scores: MODS, SOFA and LOD for predicting ICU mortality in critically ill patients. J Med Assoc Thai 2007;90:1074–81.
      OpenUrlPubMed
    63. ↵
      2. Johnson S ,
      3. Saranya A
      . Comparison of different scoring systems used in the intensive care unit. J Pulm Respir Med 2015;5.
    64. ↵
      2. Lemeshow S ,
      3. Klar J ,
      4. Teres D ,
      5. Avrunin JS ,
      6. Gehlbach SH ,
      7. Rapoport J ,
      8. Rué M
      . Mortality probability models for patients in the intensive care unit for 48 or 72 hours: a prospective, multicenter study. Crit Care Med 1994;22:1351–8.
      OpenUrlCrossRefPubMedWeb of Science
    65. ↵
      2. Lemeshow S ,
      3. Teres D ,
      4. Klar J ,
      5. Avrunin JS ,
      6. Gehlbach SH ,
      7. Rapoport J
      . Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 1993;270:2478–86.doi:10.1001/jama.1993.03510200084037
      OpenUrlCrossRefPubMedWeb of Science
    66. ↵
      2. Saklad M
      . Grading of patients for surgical procedures. Anesthesiology 1941;2:281–4.doi:10.1097/00000542-194105000-00004
      OpenUrlCrossRef
    67. ↵
      American Society of Anesthesiologists. A new classification of physical status. Anesthesiology 1963;24.
    68. ↵
      2. Vacanti CJ ,
      3. VanHouten RJ ,
      4. Hill RC
      . A statistical analysis of the relationship of physical status to postoperative mortality in 68,388 cases. Anesth Analg 1970;49:564???566–6.doi:10.1213/00000539-197007000-00010
      OpenUrl
    69. ↵
      2. Hall JC ,
      3. Hall JL
      . ASA status and age predict adverse events after abdominal surgery. J Qual Clin Pract 1996;16:102–8.
      OpenUrl
    70. ↵
      2. El-Haddawi F ,
      3. Abu-Zidan FM ,
      4. Jones W
      . Factors affecting surgical outcome in the elderly at Auckland Hospital. ANZ J Surg 2002;72:537–41.doi:10.1046/j.1445-2197.2002.02484.x
      OpenUrlPubMed
    71. ↵
      2. Hyder JA ,
      3. Reznor G ,
      4. Wakeam E ,
      5. Nguyen LL ,
      6. Lipsitz SR ,
      7. Havens JM
      . Risk prediction accuracy differs for emergency versus elective cases in the ACS-NSQIP. Ann Surg 2016;264:959–65.doi:10.1097/SLA.0000000000001558
      OpenUrl
    72. ↵
      American College of Surgeons. Surgical risk calculator. 6 Mar 2017. http://riskcalculator.facs.org/
    73. ↵
      2. Charlson ME ,
      3. Pompei P ,
      4. Ales KL ,
      5. MacKenzie CR
      . A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373–83.doi:10.1016/0021-9681(87)90171-8
      OpenUrlCrossRefPubMedWeb of Science
    74. ↵
      2. Charlson M ,
      3. Szatrowski TP ,
      4. Peterson J ,
      5. Gold J
      . Validation of a combined comorbidity index. J Clin Epidemiol 1994;47:1245–51.doi:10.1016/0895-4356(94)90129-5
      OpenUrlCrossRefPubMedWeb of Science
    75. ↵
      2. Abdullah M ,
      3. Al-Salamah SM
      . Impact of comorbidity on outcome among acute non-traumatic surgical patients. Evaluation of Charlson comorbidity index. Saudi Med J 2009;30:228–33.
      OpenUrlPubMed
    76. ↵
      2. Gabbe BJ ,
      3. Magtengaard K ,
      4. Hannaford AP ,
      5. Cameron PA
      . Is the Charlson Comorbidity Index useful for predicting trauma outcomes? Acad Emerg Med 2005;12:318–21.doi:10.1111/j.1553-2712.2005.tb01950.x
      OpenUrlCrossRefPubMedWeb of Science
    77. ↵
      2. St-Louis E ,
      3. Iqbal S ,
      4. Feldman LS ,
      5. Sudarshan M ,
      6. Deckelbaum DL ,
      7. Razek TS ,
      8. Khwaja K
      . Using the age-adjusted Charlson comorbidity index to predict outcomes in emergency general surgery. J Trauma Acute Care Surg 2015;78:318–23.doi:10.1097/TA.0000000000000457
      OpenUrl
    78. ↵
      2. Devita MA ,
      3. Bellomo R ,
      4. Hillman K ,
      5. Kellum J ,
      6. Rotondi A ,
      7. Teres D ,
      8. Auerbach A ,
      9. Chen WJ ,
      10. Duncan K ,
      11. Kenward G , et al
      . Findings of the first consensus conference on medical emergency teams. Crit Care Med 2006;34:2463–78.doi:10.1097/01.CCM.0000235743.38172.6E
      OpenUrlCrossRefPubMedWeb of Science
    79. ↵
      2. Sangji NF ,
      3. Bohnen JD ,
      4. Ramly EP ,
      5. Velmahos GC ,
      6. Chang DC ,
      7. Kaafarani HMA
      . Derivation and Validation of a Novel Physiological Emergency Surgery Acuity Score (PESAS). World J Surg 2017;41:1782–9.doi:10.1007/s00268-017-3915-9
      OpenUrl
    80. ↵
      2. Copeland GP ,
      3. Jones D ,
      4. Walters M
      . POSSUM: a scoring system for surgical audit. Br J Surg 1991;78:355–60.doi:10.1002/bjs.1800780327
      OpenUrlCrossRefPubMed
    81. ↵
      2. Whiteley MS ,
      3. Prytherch DR ,
      4. Higgins B ,
      5. Weaver PC ,
      6. Prout WG
      . An evaluation of the POSSUM surgical scoring system. Br J Surg 1996;83:812–5.doi:10.1002/bjs.1800830628
      OpenUrlPubMedWeb of Science
    82. ↵
      2. Story DA ,
      3. Fink M ,
      4. Leslie K ,
      5. Myles PS ,
      6. Yap SJ ,
      7. Beavis V ,
      8. Kerridge RK ,
      9. McNicol PL
      . Perioperative mortality risk score using pre- and postoperative risk factors in older patients. Anaesth Intensive Care 2009;37:392–8.
      OpenUrlPubMedWeb of Science
    83. ↵
      2. Hadjianastassiou VG ,
      3. Tekkis PP ,
      4. Poloniecki JD ,
      5. Gavalas MC ,
      6. Goldhill DR
      . Surgical mortality score: risk management tool for auditing surgical performance. World J Surg 2004;28:193–200.doi:10.1007/s00268-003-7174-6
      OpenUrlCrossRefPubMedWeb of Science
    84. ↵
      2. Protopapa KL ,
      3. Simpson JC ,
      4. Smith NC ,
      5. Moonesinghe SR
      . Development and validation of the Surgical Outcome Risk Tool (SORT). Br J Surg 2014;101:1774–83.doi:10.1002/bjs.9638
      OpenUrl
    85. ↵
      2. Sutton R ,
      3. Bann S ,
      4. Brooks M ,
      5. Sarin S
      . The Surgical Risk Scale as an improved tool for risk-adjusted analysis in comparative surgical audit. Br J Surg 2002;89:763–8.doi:10.1046/j.1365-2168.2002.02080.x
      OpenUrlCrossRefPubMedWeb of Science
    86. ↵
      2. Donati A ,
      3. Ruzzi M ,
      4. Adrario E ,
      5. Pelaia P ,
      6. Coluzzi F ,
      7. Gabbanelli V ,
      8. Pietropaoli P
      . A new and feasible model for predicting operative risk. Br J Anaesth 2004;93:393–9.doi:10.1093/bja/aeh210
      OpenUrlCrossRefPubMedWeb of Science
    87. ↵
      2. DeVita MA ,
      3. Smith GB ,
      4. Adam SK ,
      5. Adams-Pizarro I ,
      6. Buist M ,
      7. Bellomo R ,
      8. Bonello R ,
      9. Cerchiari E ,
      10. Farlow B ,
      11. Goldsmith D , et al
      . "Identifying the hospitalised patient in crisis"--a consensus conference on the afferent limb of rapid response systems. Resuscitation 2010;81:375–82.doi:10.1016/j.resuscitation.2009.12.008
      OpenUrlCrossRefPubMedWeb of Science
    88. ↵
      2. Buist MD ,
      3. Jarmolowski E ,
      4. Burton PR ,
      5. Bernard SA ,
      6. Waxman BP ,
      7. Anderson J
      . Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care. A pilot study in a tertiary-care hospital. Med J Aust 1999;171:22–5.
      OpenUrlPubMedWeb of Science
    89. ↵
      2. Harrison GA ,
      3. Jacques T ,
      4. McLaws ML ,
      5. Kilborn G
      . Combinations of early signs of critical illness predict in-hospital death-the SOCCER study (signs of critical conditions and emergency responses). Resuscitation 2006;71:327–34.doi:10.1016/j.resuscitation.2006.05.008
      OpenUrlCrossRefPubMedWeb of Science
    90. ↵
      2. Churpek MM ,
      3. Yuen TC ,
      4. Edelson DP
      . Risk stratification of hospitalized patients on the wards. Chest 2013;143:1758–65.doi:10.1378/chest.12-1605
      OpenUrl
    91. ↵
      2. Bertges DJ ,
      3. Goodney PP ,
      4. Zhao Y ,
      5. Schanzer A ,
      6. Nolan BW ,
      7. Likosky DS ,
      8. Eldrup-Jorgensen J ,
      9. Cronenwett JL
      . Vascular Study Group of New England. The Vascular Study Group of New England Cardiac Risk Index (VSG-CRI) predicts cardiac complications more accurately than the Revised Cardiac Risk Index in vascular surgery patients. J Vasc Surg 2010;52:674–83.doi:10.1016/j.jvs.2010.03.031
      OpenUrlCrossRefPubMedWeb of Science
    92. ↵
      2. Hillman K ,
      3. Chen J ,
      4. Cretikos M ,
      5. Bellomo R ,
      6. Brown D ,
      7. Doig G ,
      8. Finfer S ,
      9. Flabouris A
      . MERIT study investigators. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet 2005;365:2091–7.doi:10.1016/S0140-6736(05)66733-5
      OpenUrlCrossRefPubMedWeb of Science
    93. ↵
      2. Cretikos M ,
      3. Chen J ,
      4. Hillman K ,
      5. Bellomo R ,
      6. Finfer S ,
      7. Flabouris A
      . MERIT study investigators. The objective medical emergency team activation criteria: a case-control study. Resuscitation 2007;73:62–72.doi:10.1016/j.resuscitation.2006.08.020
      OpenUrlCrossRefPubMedWeb of Science
    94. ↵
      2. Morgan RJM ,
      3. Wright MM
      . An early warning system for detecting developping critical illness. Clin Intens Care 1997;8:100.
      OpenUrl
    95. ↵
      2. Gardner-Thorpe J ,
      3. Love N ,
      4. Wrightson J ,
      5. Walsh S ,
      6. Keeling N
      . The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study. Ann R Coll Surg Engl 2006;88:571–5.doi:10.1308/003588406X130615
      OpenUrlCrossRefPubMedWeb of Science
    96. ↵
      2. Subbe CP ,
      3. Kruger M ,
      4. Rutherford P ,
      5. Gemmel L
      . Validation of a modified Early Warning Score in medical admissions. QJM 2001;94:521–6.doi:10.1093/qjmed/94.10.521
      OpenUrlCrossRefPubMedWeb of Science
    97. ↵
      2. Peris A ,
      3. Zagli G ,
      4. Maccarrone N ,
      5. Batacchi S ,
      6. Cammelli R ,
      7. Cecchi A ,
      8. Perretta L ,
      9. Bechi P
      . The use of Modified Early Warning Score may help anesthesists in postoperative level of care selection in emergency abdominal surgery. Minerva Anestesiol 2012;78:1034–8.
      OpenUrlPubMed
    98. ↵
      2. Duckitt RW ,
      3. Buxton-Thomas R ,
      4. Walker J ,
      5. Cheek E ,
      6. Bewick V ,
      7. Venn R ,
      8. Forni LG
      . Worthing physiological scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study. Br J Anaesth 2007;98:769–74.doi:10.1093/bja/aem097
      OpenUrlCrossRefPubMedWeb of Science
    99. ↵
      2. Rothman MJ ,
      3. Rothman SI ,
      4. Beals J
      . Development and validation of a continuous measure of patient condition using the Electronic Medical Record. J Biomed Inform 2013;46:837–48.doi:10.1016/j.jbi.2013.06.011
      OpenUrlCrossRefPubMed
    100. ↵
      2. Tabak YP ,
      3. Sun X ,
      4. Nunez CM ,
      5. Johannes RS
      . Using electronic health record data to develop inpatient mortality predictive model: Acute Laboratory Risk of Mortality Score (ALaRMS). J Am Med Inform Assoc 2014;21:455–63.doi:10.1136/amiajnl-2013-001790
      OpenUrlCrossRefPubMed
    101. ↵
      2. Havens JM ,
      3. Peetz AB ,
      4. Do WS ,
      5. Cooper Z ,
      6. Kelly E ,
      7. Askari R ,
      8. Reznor G ,
      9. Salim A
      . The excess morbidity and mortality of emergency general surgery. J Trauma Acute Care Surg 2015;78:306–11.doi:10.1097/TA.0000000000000517
      OpenUrlCrossRefPubMed
    102. ↵
      2. Haskins IN ,
      3. Maluso PJ ,
      4. Amdur R ,
      5. Agarwal S ,
      6. Sarani B
      . Predictors of mortality after emergency general surgery: an NSQIP risk calculator. J Am Coll Surg 2016;223:S58–S59.doi:10.1016/j.jamcollsurg.2016.06.373
      OpenUrl
    103. ↵
      2. Liu Y ,
      3. Cohen ME ,
      4. Hall BL ,
      5. Ko CY ,
      6. Bilimoria KY
      . Evaluation and enhancement of calibration in the american college of surgeons NSQIP surgical risk calculator. J Am Coll Surg 2016;223:231–9.doi:10.1016/j.jamcollsurg.2016.03.040
      OpenUrlPubMed
      2. Kork F ,
      3. Balzer F ,
      4. Krannich A ,
      5. Weiss B ,
      6. Wernecke KD ,
      7. Spies C
      . Association of comorbidities with postoperative in-hospital mortality: a retrospective cohort study. Medicine 2015;94:e576.doi:10.1097/MD.0000000000000576
      OpenUrl

    Footnotes

    • Contributors Study design: JMH, MC. Literature search: JMH, ABC, CVRB, GTT, NTM, MC. Data analysis and interpretation: JMH, ABC, CVRB, GTT, NTM, MC. Writing and critical revisions: JMH, ABC, AJS, CVRB, GTT, NTM, MC.

    • Funding No funding was received for this work.

    • Competing interests None declared.

    • Patient consent Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.