Penn Medicine Sepsis Solution

Craig Umscheid, MD, MSCE

Director of the center for evidence-based practice at the university of Pennsylvania health system

Dr. Craig Umscheid is the sepsis solution clinical and project lead.

Patrick Donnelly, RN, MS, CCRN

Director of the center for evidence-based practice at the university of Pennsylvania health system

University of Pennsylvania health system resuscitation manager.

Corey Chivers, Ph.D

Senior data scientist

Corey Chivers is a senior data scientist on the penn medicine predictive healthcare team.

Inpatients who develop Sepsis can incur several additional weeks in the hospital at estimated costs totaling $30 billion annually in the United States. Sepsis strikes up to three million Americans each year.

Sever Sepsis / Septic Shock Early Warning System 2.0

The Early Warning System 2.0 is an advanced predictive system which uses patient vital signs, labs, and demographics to provide early warning of potential patient deterioration. This system uses machine learning in combination with vast amounts of individual patient data to discover patterns associated with impending patient deterioration.

The machine learning algorithm is trained on over 500 patient variables. The word cloud shows the 25 most important variables:

See below for details on the patient population, definition of severe sepsis label used to train the algorithm, details on the algorithm performance, and finally a ranking of the patient population:

Patient Population

Age > 18 years

In hospital for > 24 hours

Exclusions:

Severe Sepsis Label

We trained the algorithm to detect patients who are about to progress from a pre-septic state or septic state to severe sepsis or septic shock. To do this, we searched all patient stays between July 2011 and June 2014 (n = 162,212). To be labeled positive, patients needed to have ICD9 codes '995.92' (Severe Sepsis) or '785.52' (Septic Shock) as well as meet specific physiological criteria to define a fixed time of onset:

Physiological "Time Zero":

Blood Culture Positive AND ( Lactic Acid Level > 2.2 OR Systolic BP < 90 mm Hg)

A total of 943 patients met both the physiological criteria and were coded with Severe Sepsis or Septic Shock.

Cross Validation Performance

Sensitivity (or true positive rate): 0.79 +/- 0.07 (trained over 3507 patients, 943 positive cases)

See below for model details.

Operational Performance

We ran the model over a 3-month validation period during Oct 1st — Dec 1st 2015 which included 10,448 patient encounters. The predictive model alerted on average 10 patients a day and those patients had the following outcomes:

We also ran the model over a retrospective period encompassing July-September 2014. In this period, there were 20 cases of severe sepsis or septic shock in which the patients met both the physiological definition while not in an ICU and were coded with Severe Sepsis or Septic Shock.

Our model correctly identified 17 of these 20 patients (85%) with 10 alerts per day, compared with EWS 1.0 which identified 9 of 20 (45%) at 6 alerts per day. Additionally, our model identified these patients earlier, with a median lead time of 30.5 hours, compared to 2.5 hours for EWS 1.0.

EWS 1.0 Sensitivity = 45%, Specificity = 97%

EWS 2.0 Sensitivity = 85%, Specificity = 94%

Top 100 Most Important Features

The top 100 features by importance out of the 587 features used by the algorithm. Features include time-series inputs (eg Heart Rate (beats/min)), derived time-series statistics and trends (eg 24h Rate of Change), as well as constants (eg Height, Weight, AGE, etc).

  Feature Importance
1 BP Noninvasive Diastolic (mm Hg) 0.016243
2 BP Noninvasive Systolic (mm Hg) +++ min 24h 0.016061
3 SERVICE_DESCRIPTION >>> PULMONARY 0.015547
4 Heart Rate (beats/min) +++ RoC 24h 0.014558
5 Blood Urea Nitrogen 0.013726
6 BP Noninvasive Systolic (mm Hg) +++ minus_mean... 0.013702
7 Temperature (degrees F) 0.013580
8 Temperature (degrees F) +++ max 24h 0.013252
9 % Monocyte Automated 0.013156
10 Temperature (degrees F) +++ var 24h 0.012669
11 Blood Urea Nitrogen +++ mean 24h 0.012642
12 Heart Rate (beats/min) 0.011829
13 Blood Urea Nitrogen +++ min 24h 0.011650
14 Blood Urea Nitrogen +++ max 24h 0.011416
15 AGE 0.011090
16 BP Noninvasive Diastolic (mm Hg) +++ min 24h 0.010927
17 Carbon Dioxide 0.010580
18 Creatinine 0.010472
19 Absolute Lymphocyte Count 0.010463
20 Temperature (degrees F) +++ minus_mean 24h  0.009596
21 Temperature (degrees F) +++ RoC 24h 0.009590
22 BP Noninvasive Systolic (mm Hg) +++ var 24h  0.009372
23 Bilirubin Total 0.009272
24 Creatinine +++ mean 24h 0.008993
25 Absolute Monocyte Count 0.008889
26 Heart Rate (beats/min) +++ var 24h 0.008812
27 Creatinine +++ max 24h 0.008754
28 Heart Rate (beats/min) +++ minus_mean 24h 0.008637
29 Creatinine +++ min 24h 0.008626
30 Temperature (degrees F) +++ mean 24h 0.008606
31 BP Noninvasive Diastolic (mm Hg) +++ mean 24h 0.008346
32 BP Noninvasive Systolic (mm Hg) +++ RoC 24h 0.008200
33 Anion Gap +++ mean 24h 0.008188
34 % Neutrophil Automated 0.008091
35 Heart Rate (beats/min) +++ max 24h 0.007872
36 Red Cell Distribution Width 0.007762
37 % Lymphocyte Automated 0.007729
38 White Blood Cell 0.007547
39 Anion Gap +++ max 24h 0.007504
40 BP Noninvasive Systolic (mm Hg) +++ mean 24h 0.007416
41 White Blood Cell +++ min 24h 0.007408
42 BP Noninvasive Diastolic (mm Hg) +++ minus_mea... 0.007380
43 BP Noninvasive Diastolic (mm Hg) +++ var 24h 0.007198
44 White Blood Cell +++ mean 24h 0.007092
45 WEIGHT_LBS 0.007027
46 Alkaline Phosphatase 0.006879
47 Platelet +++ mean 24h 0.006779
48 White Blood Cell +++ max 24h 0.006759
49 SpO2 (%) +++ var 24h 0.006729
50 Platelet +++ min 24h 0.006712
51 WBC Corrected 0.006659
52 Bilirubin Direct 0.006627
53 Temperature (degrees F) +++ min 24h 0.006579
54 Anion Gap 0.006560
55 Heart Rate (beats/min) +++ mean 24h 0.006543
56 Platelet +++ max 24h 0.006529
57 BP Noninvasive Systolic (mm Hg) +++ max 24h 0.006498
58 Heart Rate (beats/min) +++ min 24h 0.006495
59 Platelet 0.006444
60 BP Noninvasive Diastolic (mm Hg) +++ max 24h 0.006376
61 Absolute Neutrophil Count 0.006362
62 Bilirubin Indirect 0.006331
63 Creatinine +++ RoC 24h 0.006263
64 Glucose Level 0.006194
65 Creatinine +++ minus_mean 24h 0.006151
66 Respirations (breaths/min) +++ minus_mean 24h 0.005874
67 Alanine Aminotransferase 0.005746
68 Anion Gap +++ min 24h 0.005675
69 Aspartate Aminotransferase 0.005656
70 SpO2 (%) +++ RoC 24h 0.005656
71 Creatinine +++ var 24h 0.005636
72 BP Noninvasive Diastolic (mm Hg) +++ RoC 24h 0.005572
73 Mean Cell Volume 0.005534
74 Respirations (breaths/min) 0.005531
75 Sodium Level 0.005489
76 White Blood Cell +++ minus_mean 24h 0.005482
77 Respirations (breaths/min) +++ var 24h 0.005474
78 FiO2 (%) 0.005463
79 Respirations (breaths/min) +++ mean 24h 0.005384
80 P CO2 Venous 0.005341
81 Prothrombin Time 0.005280
82 White Blood Cell +++ RoC 24h 0.005249
83 White Blood Cell +++ var 24h 0.005176
84 Respirations (breaths/min) +++ max 24h 0.005156
85 SpO2 (%) +++ mean 24h 0.005132
86 Calcium Level Total 0.005120
87 Respirations (breaths/min) +++ RoC 24h 0.005078
88 HEIGHT_INCHES 0.005034
89 Partial Thromboplastin Time 0.004977
90 Platelet +++ var 24h 0.004953
91 % Eosinophil Automated 0.004844
92 P O2 Venous 0.004791
93 Blood Urea Nitrogen +++ RoC 24h 0.004723
94 Phosphorus Level 0.004715
95 Red Blood Cell 0.004709
96 Potassium Level 0.004672
97 HGB HCT QC Check (Non Reportable) 0.004601
98 Hemoglobin 0.004593
99 Chloride 0.004582

Example Alert Message

73.2 year old Female admitted 2015-05-21 11:02:00 in MEDICINE

Latest:
Temperature (degrees F): 98.1
Heart Rate (beats/min): 81
BP Noninvasive Systolic (mm Hg): 127
BP Noninvasive Diastolic (mm Hg): 78
Respirations (breaths/min): 18
SpO2 (%): 96
White Blood Cell: 2.1
Creatinine: 1.7
Anion Gap: 7
Blood Urea Nitrogen: 18
Platelet: 181

View patient Chart (UPHS wifi required): http://bit.ly/1JGOUXK
Last updated at: 2015-06-10 09:24:57.558000

To reset the alert for this patient, respond to this message with "#reset".