 Good afternoon. My name is Alan Lee. I am an assistant professor in the division of gastroenterology and Hepatology at the University of Michigan I'm excited to present our work titled tidy models approaches to predicting severe Clostridioides difficile infection with multicenter cohorts. I would like to thank the organizers for inviting me to talk about our group's research today Clostridioides difficile or C. difficile is a gram-positive spore forming bacteria That is the leading cause of health care associated infections in the United States C. difficile has been identified as an urgent health threat by the Centers for Disease Control and Prevention There were an estimated 223,000 cases in hospitalized patients Leading to over 12,000 deaths and an estimated one billion dollars in health care costs in 2017 alone Approximately 8% of patients with C. difficile infection or CDI developed disease related complications These may include organ dysfunction severe sepsis Colectomy and even death Identifying patients who are at risk for severe complications from CDI may allow for the use of more aggressive therapies and potentially improve outcomes While several models have been developed to predict risk for complicated CDI Many were developed at single institutions and with small sample sizes Our group recently demonstrated that these models showed poor performance upon external validation With all of the models showing an area under the receiver operator characteristic curve or AUC under 0.7 Thus our aim was to develop a more generalizable predictive model for complicated CDI using structured electronic health record data from several geographically distinct centers in the United States and So we performed a retrospective cohort study of four cohorts in the United States Including the University of Michigan from 2010 to 2012 as well as from 2015 to 2016 University of Wisconsin from 2014 to 2015 and the University of Chicago from 2013 to 2015 All subjects who are 18 years of age or older and were diagnosed with CDI were included for analysis The primary outcome was development of complicated CDI as defined by any of the three adverse outcomes within 30 days of CDI diagnosis and this included admission to the intensive care unit or ICU Colectomy and or death that were directly attributable to CDI We used a tidy models framework for model training and validation tidy models is a collection of packages for modeling and machine learning using tidy verse principles We performed data pre-processing using the R package Miss forest a random forest based multiple imputation method for imputing missing data The recipes package from tidy models was then utilized for further pre-processing Including centering and scaling of numeric values as well as dummy coding categorical variables We also used the parsnip package for model specification Including a lasso model using glim net a random forest model using ranger and a stacked ensemble model using stacks The our sample package was then used to randomly partition the data into a training set consisting of 75% of the data and a test set consisting of the remaining 25% of the data Our sample and tune were then used to perform tenfold cross validation for estimating model accuracy and for tuning model hyper parameters Finally yardstick was used to perform model evaluation using the independent test set Here are the baseline demographics from all of the cohorts a Total of three thousand six hundred forty six patients were enrolled from all cohorts Including thirteen hundred forty-one patients from the University of Chicago eleven hundred forty-four patients from the University of Michigan twenty ten six hundred forty-six patients from the University of Michigan twenty sixteen and Five hundred fifteen patients from the University of Wisconsin The mean age from all of the cohorts was fifty eight point two years and fifty three point one percent of patients were female In addition while the vast majority of patients self identified as white race Chicago was clearly an outlier here with fifty three point two percent of patients self identifying as black furthermore All of the cohorts were comprised of nearly all inpatients except for the University of Michigan twenty sixteen Which was comprised of seventy two point six percent of inpatients Finally eight point two percent of patients from all cohorts were admitted to the ICU for non-cdi related indications while sixty one point six percent of patients were on non-cdi related antibiotics and Here are our primary end points a total of six percent of patients met the primary endpoint Including four point eight percent from the University of Chicago Seven point nine percent from the University of Michigan twenty ten four point three percent from the University of Michigan twenty sixteen and six point eight percent from the University of Wisconsin The thirty-day mortality rate across all sites was three point five percent While thirty-day collect me rates was zero point seven percent and the thirty-day ICU admission rate was two point seven percent And we found that lasso regression Random forest and stacked ensemble models all performed well when tested on an independent test set With an area under the receiver operator characteristic curve or AUC Ranging from zero point eight eight to zero point eight nine We next perform model specific variable importance analysis using the VIP package We found that variables were generally similar across models and they included peak as well as change in serum creatinine levels non-cdi related ICU admission low systolic blood pressure as well as low serum bicarbonate and albumin levels We next performed a sensitivity analysis to assess model performance and generalizability across differences in study sites Which included differences in time periods the diagnosis and management of cdi and patient demographic composition For this analysis we trained models and three cohorts and validated model performance on the fourth cohort We repeated this process three times so that each cohort served as a validation cohort once And here we found that model performance remained robust in this sensitivity analysis with AUC scores ranging from zero point eight four to zero point nine two However, we can see in the lower right-hand panel that there was a drop in performance when data from the University of Chicago Was used as the test set with AUC scores ranging from zero point seven five to zero point seven six We then tried to better understand this variability in model performance Since the definition for cdi varied by individual centers We performed an additional sensitivity analysis to determine whether diagnosis of cdi by polymerase chain reaction or PCR only versus a two-step mechanism Using a PCR screen followed by enzyme amino acid confirmation may have influenced model predictions Sites that utilized PCR only that is the University of Wisconsin and the University of Chicago Were analyzed separately from sites that used a two-step diagnostic approach That is the University of Michigan 2010 and 2016 Data were again randomly split Models were trained on 75 percent of the data and then validated on the remaining 25 percent And here we found that overall models retained good performance despite these differences in cdi diagnosis For sites that used a two-step testing approach That is the University of Michigan 2010 and 2016 shown in the right panel The models showed excellent performance with AUC scores ranging from zero point eight nine to zero point nine one However model performance was lower For sites that used PCR testing alone That is the University of Wisconsin and the University of Chicago Shown in the left panel with AUC scores ranging from zero point seven nine to zero point eight four Finally because the University of Chicago included a larger proportion of patients that self reported as black compared to other cohorts We performed a third sensitivity analysis to determine whether self reported race may affect model predictions Because of low numbers of patients identifying as Hispanic or Latino Asian American Indian or Alaska native or mixed races at each cohort Patients were grouped into white versus non-white categories Then using data from all four cohorts models were trained and validated After stratifying by race to determine if race affected model predictions And here as we could see shown in the left panel We found that model performance remained very good when trained and validated on white patients With AUC's ranging from zero point eight four to zero point eight six However model performance was noticeably worse in non-white patients as shown in the right panel With AUC's ranging from zero point six two to zero point six eight While we don't know why exactly race was an important modifier of model performance We do note that Estimates of kidney function were some of the most important predictors for complicated CDI in our models And as estimates of kidney function were initially developed using data from white patients only And prior studies have also demonstrated that black adults have higher serum creatinine levels compared with white patients We speculate that our models may have underestimated kidney function in black patients and this may have biased our results So in conclusion, we have shown using data from a large heterogeneous population Using a multicenter cohort that we were able to train and validate models that accurately predicts risk for complicated CDI Importantly the models were generalizable across centers in time However, self-reported race was an important modifier of model performance Future studies should focus on reducing disparity In model accuracy between white and non-white patients as well as improving model performance overall And perhaps inclusion of host as well as micro drive biomarkers may be one way to achieve these goals With that, I would like to thank you for your attention I'd like to thank all of my collaborators, and I'm happy to take any questions Thank you