Purpose. To validate the use of a computer program for the automatic calculation of the sequential organ failure assessment (SOFA) score, as compared to the gold standard of manual chart review. Materials and Methods. Adult admissions (age > 18 years) to the medical ICU with a length of stay greater than 24 hours were studied in the setting of an academic tertiary referral center. A retrospective cross-sectional analysis was performed using a derivation cohort to compare automatic calculation of the SOFA score to the gold standard of manual chart review. After critical appraisal of sources of disagreement, another analysis was performed using an independent validation cohort. Then, a prospective observational analysis was performed using an implementation of this computer program in AWARE Dashboard, which is an existing real-time patient EMR system for use in the ICU. Results. Good agreement between the manual and automatic SOFA calculations was observed for both the derivation ( ) and validation ( ) cohorts: 0.02 ± 2.33 and 0.29 ± 1.75 points, respectively. These results were validated in AWARE ( ). Conclusion. This EMR-based automatic tool accurately calculates SOFA scores and can facilitate ICU decisions without the need for manual data collection. This tool can also be employed in a real-time electronic environment. 1. Introduction Interest in assessment of organ dysfunction and severity of illness in the intensive care unit (ICU) setting has increased in recent years. The acute physiology and chronic health evaluation (APACHE) is one of the earliest scoring systems developed for this purpose and is currently in its fourth iteration [1, 2]. Other scoring systems have been developed, including the simplified acute physiology score (SAPS) and the mortality probability model (MPM) [3, 4]. The sequential organ failure assessment (SOFA) score is a 24-point scale designed to assess organ dysfunction and failure in critically ill patients [5]. SOFA is frequently used for the assessment of severity of illness in the ICU setting and is based on a six component, organ-based scoring system [6]. SOFA has many advantages over previously developed systems, including simplicity and ease of calibration. Traditionally, manual calculation of these scores—including SOFA—has been a time-consuming and error prone task. However, the ability of automatic tools to be used for risk assessment of patients has been demonstrated by multiple studies [7, 8]. Several automatic SOFA calculation tools have been developed. One of the earliest such tools was created and validated by
References
[1]
W. A. Knaus, J. E. Zimmerman, D. P. Wagner, E. A. Draper, and D. E. Lawrence, “APACHE-acute physiology and chronic health evaluation: a physiologically based classification system,” Critical Care Medicine, vol. 9, no. 8, pp. 591–597, 1981.
[2]
J. E. Zimmerman, A. A. Kramer, D. S. McNair, and F. M. Malila, “Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients,” Critical Care Medicine, vol. 34, no. 5, pp. 1297–1310, 2006.
[3]
J.-R. Le Gall, P. Loirat, and A. Alperovitch, “A simplified acute physiology score for ICU patients,” Critical Care Medicine, vol. 12, no. 11, pp. 975–977, 1984.
[4]
S. Lemeshow, D. Teres, and H. Pastides, “A method for predicting survival and mortality of ICU patients using objectively derived weights,” Critical Care Medicine, vol. 13, no. 7, pp. 519–525, 1985.
[5]
J.-L. Vincent, R. Moreno, J. Takala et al., “The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure,” Intensive Care Medicine, vol. 22, no. 7, pp. 707–710, 1996.
[6]
F. Lopes Ferreira, D. Peres Bota, A. Bross, C. Mélot, and J.-L. Vincent, “Serial evaluation of the SOFA score to predict outcome in critically ill patients,” Journal of the American Medical Association, vol. 286, no. 14, pp. 1754–1758, 2001.
[7]
A. Alsara, D. O. Warner, G. Li, V. Herasevich, O. Gajic, and D. J. Kor, “Derivation and validation of automated electronic search strategies to identify pertinent risk factors for postoperative acute lung injury,” Mayo Clinic Proceedings, vol. 86, no. 5, pp. 382–388, 2011.
[8]
S. Chandra, D. Agarwal, A. Hanson et al., “The use of an electronic medical record based automatic calculation tool to quantify risk of unplanned readmission to the intensive care unit: a validation study,” Journal of Critical Care, vol. 26, no. 6, pp. 634.e9–634.e15, 2011.
[9]
A. Junger, J. Engel, M. Benson et al., “Discriminative power on mortality of a modified sequential organ failure assessment score for complete automatic computation in an operative intensive care unit,” Critical Care Medicine, vol. 30, no. 2, pp. 338–342, 2002.
[10]
J. M. Engel, A. Junger, M. Zimmer et al., “Correlation of a computerized SOFA score and derived measures with length of stay at an operative ICU,” Anasthesiologie Intensivmedizin Notfallmedizin Schmerztherapie, vol. 38, no. 6, pp. 397–402, 2003.
[11]
J. L. Nates, M. Cárdenas-Turanzas, C. Wakefield et al., “Automating and simplifying the SOFA score in critically ill patients with cancer,” Health Informatics Journal, vol. 16, no. 1, pp. 35–47, 2010.
[12]
J. L. Nates, M. Cárdenas-Turanzas, J. Ensor, C. Wakefield, S. K. Wallace, and K. J. Price, “Cross-validation of a modified score to predict mortality in cancer patients admitted to the intensive care unit,” Journal of Critical Care, vol. 26, no. 4, pp. 388–394, 2011.
[13]
D. A. Zygun, K. B. Laupland, G. H. Fick, J. D. Sandham, and C. J. Doig, “Limited ability of SOFA and MOD scores to discriminate outcome: a prospective evaluation in 1,436 patients,” Canadian Journal of Anesthesia, vol. 52, no. 3, pp. 302–308, 2005.
[14]
M. Thomas, C. Bourdeaux, Z. Evans, D. Bryant, R. Greenwood, and T. Gould, “Validation of a computerised system to calculate the sequential organ failure assessment score,” Intensive Care Medicine, vol. 37, no. 3, p. 557, 2011.
[15]
V. Herasevich, B. W. Pickering, Y. Dong, S. G. Peters, and O. Gajic, “Informatics infrastructure for syndrome surveillance, decision support, reporting, and modeling of critical illness,” Mayo Clinic Proceedings, vol. 85, no. 3, pp. 247–254, 2010.
[16]
D. Blumenthal, C. DesRoches, K. Donelan, et al., Health Information Technology in the United States: The Information Base for Progress, Robert Wood Johnson Foundation, 2006.
[17]
A. K. Jha, C. M. DesRoches, E. G. Campbell, et al., “Use of electronic health records in U.S. hospitals,” The New England Journal of Medicine, vol. 360, no. 16, pp. 1628–1638, 2009.
[18]
B. W. Pickering, V. Herasevich, A. Ahmed, and O. Gajic, “Novel representation of clinical information in the ICU—developing user interfaces which reduce information overload,” Applied Clinical Informatics, vol. 1, no. 2, pp. 116–131, 2010.
[19]
A. Ahmed, S. Chandra, V. Herasevich, O. Gajic, and B. W. Pickering, “The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance,” Critical Care Medicine, vol. 39, no. 7, pp. 1626–1634, 2011.
[20]
J. M. Bland and D. G. Altman, “Statistical methods for assessing agreement between two methods of clinical measurement,” The Lancet, vol. 1, no. 8476, pp. 307–310, 1986.
[21]
P. S. Myles and J. Cui, “I. Using the Bland-Altman method to measure agreement with repeated measures,” British Journal of Anaesthesia, vol. 99, no. 3, pp. 309–311, 2007.
[22]
L. M. Chen, C. M. Martin, T. L. Morrison, and W. J. Sibbald, “Interobserver variability in data collection of the APACHE II score in teaching and community hospitals,” Critical Care Medicine, vol. 27, no. 9, pp. 1999–2004, 1999.
[23]
M. Suistomaa, A. Kari, E. Ruokonen, and J. Takala, “Sampling rate causes bias in APACHE II and SAPS II scores,” Intensive Care Medicine, vol. 26, no. 12, pp. 1773–1778, 2000.
[24]
T. W. Felton, R. Sander, M. Al-Aloul, P. Dark, and A. M. Bentley, “Can a score derived from the critical care minimum data set be used as a marker of organ dysfunction? A pilot study,” BMC Research Notes, vol. 2, article 77, 2009.
[25]
B. A. Stewart, S. Fernandes, E. Rodriguez-Huertas, and M. Landzberg, “A preliminary look at duplicate testing associated with lack of electronic health record interoperability for transferred patients,” Journal of the American Medical Informatics Association, vol. 17, no. 3, pp. 341–344, 2010.
[26]
A. Wright, D. F. Sittig, J. S. Ash, S. Sharma, J. E. Pang, and B. Middleton, “Clinical decision support capabilities of commercially-available clinical information systems,” Journal of the American Medical Informatics Association, vol. 16, no. 5, pp. 637–644, 2009.
