Shimabukuro, D. | bioRxiv Science

bioRxiv2018

Multicenter validation of a machine learning algorithm for 48 hour all-cause mortality prediction

PurposeThis study evaluates a machine-learning-based mortality prediction tool.\n\nMaterials and MethodsWe conducted a retrospective study with data drawn from three academic health centers. Inpatients of at least 18 years of age and with at least one observation of each vital sign were included. Predictions were made at 12, 24, and 48 hours before death. Models fit to training data from each institution were evaluated on hold-out test data from the same institution and data from the remaining institutions. Predictions were compared to those of qSOFA and MEWS using area under the receiver operating characteristic curve (AUROC).\n\nResultsFor training and testing on data from a single institution, machine learning predictions averaged AUROCs of 0.97, 0.96, and 0.95 across institutional test sets for 12-, 24-, and 48-hour predictions, respectively. When trained and tested on data from different hospitals, the algorithm achieved AUROC up to 0.95, 0.93, and 0.91, for 12-, 24-, and 48-hour predictions, respectively. MEWS and qSOFA had average 48-hour AUROCs of 0.86 and 0.82, respectively.\n\nConclusionThis algorithm may help identify patients in need of increased levels of clinical care.

bioinformatics↗

bioRxiv2018

Multicenter validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU

ObjectivesWe validate a machine learning-based sepsis prediction algorithm (InSight) for detection and prediction of three sepsis-related gold standards, using only six vital signs. We evaluate robustness to missing data, customization to site-specific data using transfer learning, and generalizability to new settings.\n\nDesignA machine learning algorithm with gradient tree boosting. Features for prediction were created from combinations of only six vital sign measurements and their changes over time.\n\nSettingA mixed-ward retrospective data set from the University of California, San Francisco (UCSF) Medical Center (San Francisco, CA) as the primary source, an intensive care unit data set from the Beth Israel Deaconess Medical Center (Boston, MA) as a transfer learning source, and four additional institutions datasets to evaluate generalizability.\n\nParticipants684,443 total encounters, with 90,353 encounters from June 2011 to March 2016 at UCSF.\n\nInterventionsnone\n\nPrimary and secondary outcome measuresArea under the receiver operating characteristic curve (AUROC) for detection and prediction of sepsis, severe sepsis, and septic shock.\n\nResultsFor detection of sepsis and severe sepsis, InSight achieves an area under the receiver operating characteristic (AUROC) curve of 0.92 (95% CI 0.90 - 0.93) and 0.87 (95% CI 0.86 - 0.88), respectively. Four hours before onset, InSight predicts septic shock with an AUROC of 0.96 (95% CI 0.94 -0.98), and severe sepsis with an AUROC of 0.85 (95% CI 0.79 - 0.91).\n\nConclusionsInSight outperforms existing sepsis scoring systems in identifying and predicting sepsis, severe sepsis, and septic shock. This is the first sepsis screening system to exceed an AUROC of 0.90 using only vital sign inputs. InSight is robust to missing data, can be customized to novel hospital data using a small fraction of site data, and retained strong discrimination across all institutions.\n\nStrengths and limitations of this studyO_LIMachine learning is applied to the detection and prediction of three separate sepsis standards in the emergency department, general ward and intensive care settings.\nC_LIO_LIOnly six commonly measured vital signs are used as input for the algorithm.\nC_LIO_LIThe algorithm is robust to randomly missing data.\nC_LIO_LITransfer learning successfully leverages large dataset information to a target dataset.\nC_LIO_LIRetrospective nature of the study does not predict clinician reaction to information.\nC_LI

bioinformatics↗