糖心vlog

Key Points

Question听 Can postoperative intensive care unit (ICU) admission be accurately predicted using the 8 preoperative variables of the Surgical Risk Preoperative Assessment System?

Findings听 In this decision analytical model study including data from 34 568 patients, the Surgical Risk Preoperative Assessment System accurately predicted postoperative ICU admission.

Meaning听 Predicting postoperative ICU admission allows planning for use of a valuable and limited resource, particularly in settings where ICU beds are scarce.

Importance听 Despite limited capacity and expensive cost, there are minimal objective data to guide postoperative allocation of intensive care unit (ICU) beds. The Surgical Risk Preoperative Assessment System (SURPAS) uses 8 preoperative variables to predict many common postoperative complications, but it has not yet been evaluated in predicting postoperative ICU admission.

Objective听 To determine if the SURPAS model could accurately predict postoperative ICU admission in a broad surgical population.

Design, Setting, and Participants听 This decision analytical model was a retrospective, observational analysis of prospectively collected patient data from the 2012 to 2018 American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) database, which were merged with individual patients鈥� electronic health record data to capture postoperative ICU use. Multivariable logistic regression modeling was used to determine how the 8 preoperative variables of the SURPAS model predicted ICU use compared with a model inputting all 28 preoperatively available NSQIP variables. Data included in the analysis were collected for the ACS NSQIP at 5 hospitals (1 tertiary academic center, 4 academic affiliated hospitals) within the University of Colorado Health System between January 1, 2012, and December 31, 2018. Included patients were those undergoing surgery in 9 surgical specialties during the 2012 to 2018 period. Data were analyzed from May 29 to July 30, 2021.

Exposure听 Surgery in 9 surgical specialties, including general, gynecology, orthopedic, otolaryngology, plastic, thoracic, urology, vascular, and neurosurgery.

Main Outcomes and Measures听 Use of ICU care up to 30 days after surgery.

Results听 A total of 34鈥�568 patients were included in the analytical data set: 32鈥�032 (92.7%) in the cohort without postoperative ICU use and 2545 (7.4%) in the cohort with postoperative ICU use (no ICU use: mean [SD] age, 54.9 [16.6] years; 18 188 women [56.8%]; ICU use: mean [SD] age, 60.3 [15.3] years; 1333 men [52.4%]). For the internal chronologic validation of the 7-variable SURPAS model, data from 2012 to 2016 were used as the training data set (n鈥�=鈥�24鈥�250, 70.2% of the total sample size of 34鈥�568) and data from 2017 to 2018 were used as the test data set (n鈥�=鈥�10鈥�318, 29.8% of the total sample size of 34鈥�568). The C statistic improved in the test data set compared with the training data set (0.933; 95% CI, 0.924-0.941 vs 0.922; 95% CI, 0.917-0.928), whereas the Brier score was slightly worse in the test data set compared with the training data set (0.045; 95% CI, 0.042-0.048 vs 0.045; 95% CI, 0.043-0.047). The SURPAS model compared favorably with the model inputting all 28 NSQIP variables, with both having good calibration between observed and expected outcomes in the Hosmer-Lemeshow graphs and similar Brier scores (model inputting all variables,鈥�0.044; 95% CI, 0.043-0.048; SURPAS model,鈥�0.045; 95% CI, 0.042-0.046) and C statistics (model inputting all variables,鈥�0.929; 95% CI, 0.925-0.934; SURPAS model,鈥�0.925; 95% CI, 0.921-0.930).

Conclusions and Relevance听 Results of this decision analytical model study revealed that the SURPAS prediction model accurately predicted postoperative ICU use across a diverse surgical population. These results suggest that the SURPAS prediction model can be used to help with preoperative planning and resource allocation of limited ICU beds.

Intensive care unit (ICU) beds are an expensive and limited resource, which was particularly demonstrated during the COVID-19 pandemic.^1-4 With roughly 10% of surgical patients requiring postoperative ICU care⁵ and 22 million surgical procedures in the US annually,⁶ there are more than 2 million postoperative ICU admissions each year. Unanticipated surges in ICU admissions, like those experienced during the COVID-19 pandemic, present problems for patients, health care professionals, and hospital systems alike.³

Accurate preoperative prediction of surgical patients requiring postoperative ICU care could prove useful in allocating this precious resource. We identified 26 studies^7-32 over the past 20 years related to predicting postoperative ICU use. Most of the studies (77%) covered specific types of surgery and thus were not generalizable to broad surgical populations.^7-26 Six studies^27-32 included broader surgical populations, but 3 studies^27-29 had small sample sizes (100 to 6000 patients), and 1 study included both preoperative and intraoperative predictors, making it not useful for prediction in the preoperative period.³⁰ One study³¹ from Singapore General Hospital developed a surgical risk calculator for the prediction of postsurgical mortality and ICU admission in 79鈥�914 noncardiac and nonneurosurgical patients that could be useful for a broad surgical population.³² There have been a paucity of studies using the data collected by the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP), probably because the database does not include ICU admission as an outcome. Kongkaewpaisan and colleagues²⁰ used the ACS NSQIP database to predict ICU need, which was a composite of outcomes that probably required ICU admission, but this does not necessarily equate to actual ICU use.

Meguid and colleagues^33-35 and Henderson et al³⁶ developed the Surgical Risk Preoperative Assessment System (SURPAS), which is a universal surgical risk calculator that uses 8 preoperative variables to estimate the risk of 12 postoperative adverse outcomes.^37-39 The models were developed using the large ACS NSQIP database encompassing more than 6 million surgical cases from 9 surgical specialties. However, postoperative ICU use is currently not one of the outcomes predicted by SURPAS. The purpose of this study was to determine if the 8-variable SURPAS risk model accurately estimates postoperative ICU admission.

This was a retrospective, observational, decision analytical model study of the prospectively collected ACS NSQIP database from January 1, 2012, to December 31, 2018. Because the ACS NSQIP does not collect postoperative ICU use, we restricted the study to the University of Colorado Health System (UCHealth) ACS NSQIP data so that we could obtain postoperative ICU use from the local electronic health record software, Epic (Epic Systems Corporation). The UCHealth System hospitals participating in the ACS NSQIP included the University of Colorado Hospital and 4 academic-affiliated hospitals in the Colorado Springs and Fort Collins metropolitan areas of the state. This research was approved by the Colorado Multiple Institutional Review Board with a waiver of informed consent owing to the use of deidentified patient data. The Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis () reporting guidelines were followed in the analysis and reporting of this study.

The ACS NSQIP collects a systematic sample of surgical cases in 9 specialty areas (general, gynecology, orthopedic, otolaryngology, plastic, thoracic, urology, vascular, and neurosurgery). Data collection for each surgical case includes preoperative demographic characteristics and comorbidities, selected variables about the operation, and 30-day postoperative outcomes. Data are collected by trained, certified, surgical clinical reviewers and are periodically audited for accuracy and completeness. NSQIP collects 40 preoperative variables, including 28 patient characteristics and comorbidities and 12 laboratory values. Previous work has shown that the laboratory variables do not add to the predictive ability of the 28 patient characteristics and comorbidities³⁴; therefore, these were not used in the analysis. The race and ethnicity variables came from the ACS NSQIP database. These were self-assigned or assigned by institutional personnel per the practice of the institution. Race and ethnicity were used to characterize the sample and to study their association with postoperative ICU use. The specific races and ethnicities identified included American Indian or Alaska Native, Asian or Pacific Islander, non-Hispanic Black, Hispanic, non-Hispanic White, or unknown if that information was missing from the patient鈥檚 record. The 8 preoperative variables included in the SURPAS prediction model were 4 related to the operation (Current Procedural Terminology [CPT]鈥搒pecific event rate of the outcome of interest calculated from the ACS NSQIP database, work relative value unit [RVU] of the primary operation, specialty of the primary surgeon, and inpatient/outpatient setting) and 4 related to the patient (age, American Society of Anesthesiology [ASA] physical status classification I-V, functional health status, and emergency status). In this study, CPT-specific event rate was not used because it was believed that the estimates using only UCHealth data would not be very accurate owing to small sample size and data origin from only 1 health care system.

Patient records in the UCHealth ACS NSQIP sample (2012-2018) were matched to their electronic health record data from Epic using the patient medical record numbers and operative dates to obtain 30-day postoperative ICU use (yes or no). The data and linkage were provided by the University of Colorado鈥檚 Health Data Compass Data Warehouse project.

If a patient had 1 or more operations in the database, only the first operation was used. Patients were also excluded if they had any missing variables (complete-case analysis). An analysis of the bivariable association of each of the 28 preoperative predictor variables with postoperative ICU use was performed using a 蠂² test for categorical predictor variables or an unpaired t test for continuous variables.

A stepwise forward-selection logistic regression analysis was performed including all 28 ACS NSQIP preoperative predictor variables, using P values of .05 for entry into and exiting the prediction model. All P values were 2-sided with significance set as P < .05. The established SURPAS model was then performed using 7 of the 8 SURPAS preoperative predictor variables. The 2 models were compared for performance using the C statistic for discrimination, Hosmer-Lemeshow (H-L) for goodness of fit, and Brier scores for overall fit. In the SURPAS model, variables were ordered by their 蠂² test statistic and not by stepwise forward selection.

An internal chronologic validation of the 7-variable SURPAS model was done by temporally splitting the data into a training set (2012-2016) and test set (2017-2018). The model was developed using the training set and applied to both the training and the test data sets. The C statistic, H-L graphs, and Brier scores were then compared between the training and test data sets. All statistical analyses and the H-L graphs were performed from May 29 to July 30, 2021, using SAS, version 9.4 (SAS Institute).

There were 35鈥�073 observations in the UCHealth NSQIP database and 34鈥�587 unique patient identification numbers. For those patients with more than 1 operation, the first operation was selected. Nineteen patients were excluded for missing work RVU, resulting in a total of 34鈥�568 patients: 32鈥�032 (92.7%) in the cohort without postoperative ICU use and 2545 (7.4%) in the cohort with postoperative ICU use (no ICU use: mean [SD] age, 54.9 [16.6] years; 18 188 women [56.8%]; 13 835 men [43.2%]; ICU use: mean [SD] age, 60.3 [15.3] years; 1212 women [47.6%]; 1333 men [52.4%]) in the analytical data set. In the group without ICU use, 98 patients (0.3%) were identified as American Indian or Alaska Native, 428 (1.3%) as Asian or Pacific Islander, 1196 (3.7%) as non-Hispanic Black, 3143 (9.8%) as Hispanic, 25 625 (80.0%) as non-Hispanic White, and 1533 (4.8%) as unknown race. In the group with ICU use, 11 patients (0.4%) were identified as American Indian or Alaska Native, 30 (1.2%) as Asian or Pacific Islander, 120 (4.7%) as non-Hispanic Black, 234 (9.2%) as Hispanic, 2023 (79.5%) as non-Hispanic White, and 127 (5.0%) as unknown race.

eFigure 1 in the Supplement presents the frequency distribution of days from operation to ICU admission for the 2545 patients admitted to the ICU: 1467 of 2545 patients (57.7%) were admitted on the day of surgery (postoperative day [POD] 0), 423 of 2545 (16.6%) on POD 1 (cumulative 74% within 1 day after surgery), 2172 of 2545 (85.2%) within POD 4, and 2309 of 2545 (90.6%) within POD 8. Table 1 presents the bivariable association between the 28 ACS NSQIP preoperative characteristics and postoperative ICU use. The group with postoperative ICU use had more men than women (1333 [8.8%] vs 1212 [6.2%]), more patients with underweight (113 [17.7%]) than patients with normal weight (777 [8.0%]) or patients with overweight (752 [6.7%]), and patients with each of 17 different comorbidities. The group with postoperative ICU use had more patients who were partially (188 [25.4%]) or totally (25 [28.7%]) dependent than independent (2332 [6.9%]) of activities of daily living, more patients who were transferred from another acute care hospital (349 [38.2%]) or chronic care facility (34 [23.1%]) than from home (2162 [6.4%]) before surgery, more inpatients (2501 [13.8%]) than outpatients (44 [0.3%]), more patients undergoing emergency operations (385 [16.6%]) than not (2160 [6.7%]), more patients with higher ASA class than lower ASA class (I: 13 [0.4%]; II: 577 [3.3%]; III: 1460 [11.7%]; IV: 468 [33.6%]; V: 27 [84.4%]), and more patients undergoing more complex operations than in the group that did not use postoperative ICU (ICU use: mean [SD] work RVU, 26.1鈥塠12.0] units vs no ICU use: mean [SD] work RVU, 15.6鈥塠7.6] units). There were also large differences in ICU use by surgical specialty: the highest users were in vascular surgery, 46.9% (n = 328); neurosurgery, 28.0% (n = 732); and thoracic surgery, 26.5% (n = 242). ICU use in the other 6 specialties ranged from 0.9% (n = 35) in gynecology to 8.1% (n = 789) in general surgery.

Table 2 presents the stepwise logistic regression analysis using the 28 ACS NSQIP preoperative predictor variables to predict postoperative ICU use. Twelve of the 28 variables were statistically significant predictors. The first 4 variables (3 related to the operation: surgical specialty, work RVU, inpatient/outpatient operation; and 1 related to the patient: ASA class) accounted for 98.8% (C statistic,鈥�0.923; 95% CI, 0.918-0.927) of the total C statistic of the 28-variable model (C statistic,鈥�0.934; 95% CI, 0.930-0.939). Figure 1, which displays the H-L graphs of observed and expected events (applying the 12-variable model to the full data set), showed good calibration. The Brier score was 0.044 (95% CI, 0.043-0.048).

Table 3 presents the 7-variable SURPAS model. The total C statistic was 0.925 (95% CI, 0.921-0.930), just slightly smaller than the C statistic of the 12-variable model of 0.929 (95% CI, 0.925-0.934). The H-L graphs of observed and expected events (applying the SURPAS model to the full data set) also showed good calibration (Figure 2). The Brier score was 0.045 (95% CI, 0.042-0.046), just slightly larger than the Brier score of the 12-variable model.

For the internal chronologic validation of the 7-variable SURPAS model, 2012-2016 were used as the training data set (24鈥�250 of 34 568 [70.2%]) and 2017-2018 were used as the test data set (10鈥�318 of 34 568 [29.8%]). eFigure 2 in the Supplement shows the SURPAS model applied to the training data set, and eFigure 3 in the Supplement shows the SURPAS model applied to the test data set. The C statistic improved in the test data set compared with the training data set (0.933; 95% CI, 0.924-0.941 vs 0.922; 95% CI, 0.917-0.928), whereas the Brier score was slightly worse in the test data set compared with the training data set (0.045; 95% CI, 0.042-0.048 vs 0.045; 95% CI, 0.043-0.047). These results showed good internal chronologic validation of the SURPAS model.

We also examined the agreement between actual ICU use and the ICU need composite variable as used by Kongkaewpaisan and colleagues.²⁰ Of 443 patients who were classified as needing ICU care, 371 of 443 (83.8%) actually had ICU use; therefore, the composite outcome variable did identify very sick patients, most of whom ended up in the ICU. However, the composite variable also missed a large percentage of the patients who did use the ICU (2178 of 2549鈥塠85.4%]).

This decision analytical model study demonstrated that the 8-variable SURPAS model had excellent discrimination (C statistic, 0.925; 95% CI, 0.921-0.930) and calibration (H-L graphs) in predicting a postoperative ICU stay in a broad surgical population. It also performed well in an internal chronologic validation test, with no important changes in the C statistic, Brier score, or H-L graphs between the model applied to the training and test data sets.

The results of this study are unique because most of the literature on predictors of postoperative ICU stay has been done for specific operations and not for broad surgical populations. In the 6 studies^27-32 using broad surgical populations, 3 studies^27-29 had relatively small sample sizes (100-6000 patients), and 1 study³⁰ was done using both preoperative and intraoperative variables. We found only one other research group, from the Singapore General Hospital, that used a large, broad surgical population^31,32 to develop a preoperative prediction model, the Combined Assessment of Risk Encountered in Surgery (CARES) model, for postoperative ICU use. In the CARES study, 4 of 7 predictor variables were identical or similar to the SURPAS variables (patient age, ASA class, emergency surgery, and a measure of surgical complexity). In addition, the CARES study used anemia status, sex, and congestive heart failure, which were not used in SURPAS, whereas SURPAS used functional health status of the patient, surgeon specialty, and inpatient/outpatient setting, which the CARES model did not use. The magnitudes of the C statistic values also differed; the SURPAS model had a C statistic of 0.925 (95% CI, 0.921-0.930) compared with a C statistic of 0.837 (95% CI, 0.808-0.868) in the CARES model. Another factor that makes our study unique is that the ACS鈥� surgical risk calculator, which is also based on the ACS NSQIP data set and probably the most widely used of the surgical risk calculators, does not have ICU stay as one of its outcome variables.⁴⁰

We believe that the SURPAS prediction model performed at the patient鈥檚 preoperative visit could be useful for surgeon and patient decision-making and for efficient planning of the use of this limited and expensive resource. One use of the prediction model might be to identify elective surgical patients at high risk for a postoperative ICU stay who might have their surgery delayed when ICU resources are scarce, eg, as in the COVID-19 pandemic. We recently published a report on using SURPAS to identify the high-risk surgical patient, which may aid in this application.⁴¹ Another potential use might be the estimation of the number of ICU beds needed for operating on a group of surgical patients. In this case, one would need the SURPAS variables for the group as well as the timing after surgery when the ICU beds might be needed and for how long they would be occupied.

The characteristics of the SURPAS model, including the minimal number of predictor variables, which are well known at the preoperative visit and cover a broad surgical population, should make the tool attractive to health care professionals. We have made the logistic regression equations in predictive model markup language code freely available for others to use,⁴² we have provided SURPAS online,⁴³ and we are developing a mobile application of the SURPAS risk prediction tool in order that health care professionals at other institutions can use the tool. In the SURPAS online tool, 7 variables are entered (CPT code of primary operation, patient age, functional health status, ASA class, surgeon specialty, emergency operation [yes, no], and inpatient/outpatient setting), and then risks of all outcomes are calculated. There is a search bar that uses a CPT code search engine when the user types the name of the operation. Once the CPT code is specified, there is a table lookup for the variables work RVU and CPT-specific event rate for each outcome (not used for the ICU use outcome, as previously mentioned). Two additional variables (tier and the need for an ICU bed) are collected for purposes of triaging surgical patients during the COVID-19 pandemic in the UCHealth System but are not used in calculating the SURPAS risks.

We also found that, although most of the patients (371 of 443 [83.8%]) who experienced the cluster of ACS NSQIP outcomes to label them as positive for ICU need did have a postoperative ICU stay, this surrogate outcome also missed 85% (2178 of 2549) of the patients who did use the ICU postoperatively. Reasons for this poor agreement between ICU need and actual ICU use might include that the ICU need variable does not include several important indications for admission to intensive care, including hourly neurologic checks, hourly neurovascular checks, use of certain medication infusions, or other nursing interventions required hourly. Additionally, postoperative ICU admission often is a complex decision that surgeons make that varies depending on factors other than explicit admission indications, like surgeon experience, practice setting, operative course, and comfort. Thus, this outcome being used in the literature²⁰ is not a good substitute for actual ICU use.

The inclusion of data from several hospitals representing different hospital types strengthens the applicability of our findings to other hospital systems. A major limitation of this study was that it was from a single health care system, although the UCHealth ACS NSQIP and electronic health record data did come from 5 separate hospitals, 1 being a major tertiary university hospital and 4 smaller community hospitals in the same state. Based on local practice patterns, we suspect that patients who were transferred to ICU immediately after surgery were usually provided a bed based on surgeon request and less frequently based on intraoperative physiology changes necessitating increased monitoring. However, for the patients transferred to an ICU on POD 1 or greater, this would have been due to some change in patient status that required increased monitoring or support. Local variation in practice patterns at each of the participating hospitals dictates this change in level of care, and we cannot control for this subjective variability. It would be advantageous to repeat this study in the future using multiple health care systems from different states in the US. There are likely institutional variations in ICU use, which are based on surgeon and anesthetist experience and preferences, and do not necessarily conform to strict indicators for ICU need.

This decision analytical model study found that the 8-variable SURPAS risk prediction model performed well at predicting postoperative ICU use in a broad surgical population across a diverse hospital system. This tool may be of use to institutions where ICU bed resources are limited, especially during circumstances such as the COVID-19 pandemic.

Accepted for Publication: December 7, 2021.

Published Online: February 16, 2022. doi:10.1001/jamasurg.2021.7580

Corresponding Author: Robert A. Meguid, MD, MPH, Department of Surgery, University of Colorado School of Medicine, Anschutz Medical Campus, 12631 E 17th Ave, C-310, Aurora, CO 80045 (robert.meguid@cuanschutz.edu).

Author Contributions: Drs Rozeboom and Meguid had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Rozeboom, Henderson, Dyas, Colborn, Hammermeister, McIntyre, Meguid.

Acquisition, analysis, or interpretation of data: Rozeboom, Dyas, Bronsert, Colborn, Lambert-Kerzner, Hammermeister, Meguid.

Drafting of the manuscript: Rozeboom, Henderson, Dyas, Meguid.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Rozeboom, Henderson, Bronsert, Meguid.

Obtained funding: Dyas, Meguid.

Administrative, technical, or material support: Colborn, McIntyre, Meguid.

Supervision: Henderson, McIntyre, Meguid.

Conflict of Interest Disclosures: Dr McIntyre reported receiving a research grant from Genentech during the conduct of the study. No other disclosures were reported.

Funding/Support: This work was supported by Surgical Outcomes and Applied Research program funding from the Department of Surgery, University of Colorado School of Medicine.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The American College of Surgeons National Surgical Quality Improvement Program and participating hospitals are the source of these data; they have not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.