- Research article
- Open Access
Can we predict early 7-day readmissions using a standard 30-day hospital readmission risk prediction model?
BMC Medical Informatics and Decision Making volume 20, Article number: 227 (2020)
Despite focus on preventing 30-day readmissions, early readmissions (within 7 days of discharge) may be more preventable than later readmissions (8–30 days). We assessed how well a previously validated 30-day EHR-based readmission prediction model predicts 7-day readmissions and compared differences in strength of predictors.
We conducted an observational study on adult hospitalizations from 6 diverse hospitals in North Texas using a 50–50 split-sample derivation and validation approach. We re-derived model coefficients for the same predictors as in the original 30-day model to optimize prediction of 7-day readmissions. We then compared the discrimination and calibration of the 7-day model to the 30-day model to assess model performance. To examine the changes in the point estimates between the two models, we evaluated the percent changes in coefficients.
Of 32,922 index hospitalizations among unique patients, 4.4% had a 7-day admission and 12.7% had a 30-day readmission. Our original 30-day model had modestly lower discrimination for predicting 7-day vs. any 30-day readmission (C-statistic of 0.66 vs. 0.69, p ≤ 0.001). Our re-derived 7-day model had similar discrimination (C-statistic of 0.66, p = 0.38), but improved calibration. For the re-derived 7-day model, discharge day factors were more predictive of early readmissions, while baseline characteristics were less predictive.
A previously validated 30-day readmission model can also be used as a stopgap to predict 7-day readmissions as model performance did not substantially change. However, strength of predictors differed between the 7-day and 30-day model; characteristics at discharge were more predictive of 7-day readmissions, while baseline characteristics were less predictive. Improvements in predicting early 7-day readmissions will likely require new risk factors proximal to day of discharge.
Despite intense focus on preventing 30-day readmissions, early readmissions within the first 7 days of hospital discharge may be more preventable than later readmissions (8–30 days post-discharge) [1,2,3,4,5,6,7]. Early readmissions are more closely related to potential gaps in care during the index hospitalization  or reflect premature discharge . Identifying patients at risk for early, rather than later readmissions may be a more effective strategy to tailor resource-intensive transitional care interventions to prevent readmissions. However, current risk prediction models often only identify patients at risk for 30-day readmission [8,9,10] and often fail to use electronic health record (EHR) data effectively to allow for real-time operationalization of the model [10, 11]. There is a paucity of research developing prediction models for adult 7-day readmissions [8, 12], which may be due to federal financial penalties tied to 30-day readmissions. Yet, to our knowledge, no study has investigated if a 30-day model can be reapplied effectively to predict the important subset of 7-day readmissions. Therefore, we assessed how well a validated 30-day EHR-based readmission risk prediction model  would predict early 7-day readmissions, and whether there were differences in the strength of predictors for 7-day versus 30-day readmissions.
We conducted an observational cohort study of consecutive hospitalizations by adults ≥18 years from November 2009 to October 2010 using electronic health record (EHR) data from 6 diverse hospitals in north Texas, including safety-net, academic, and community sites. We included index hospitalizations of patients who were alive 30 days post-discharge. Patients who died in the hospital, were transferred to another hospital, or left against medical advice were excluded. The primary outcome was all-cause non-elective 7-day hospital readmissions within a 100-mile radius of Dallas, Texas (includes 75 acute care hospitals), which were retrieved from an all-payer regional hospitalization database. The previously validated 30-day readmission multivariate logistic regression model was developed using a 50–50 derivation-validation split. Multiple groups of candidate predictors including socioeconomic, admission day, hospital stay, and discharge day variables were included in development of the model. All variables were available in the EHR for all participating hospitals and were plausible based on previous literature and clinical expertise. Further details about model development have been previously published .
Using the same derivation cohort, we ran a multivariate logistic regression to re-derive model coefficients for the same predictors from our validated 30-day readmission model  (also developed from the same cohort) to optimize prediction of 7-day readmissions. Using the same cohort allows for direct comparison of model performance and changes in coefficient direction and magnitude between models. We used the validation cohort to compare the discrimination and calibration of our 7-day readmission model with our original 30-day model to predict 7-day readmissions. Discrimination was assessed using the C statistic, which measures the goodness of fit of a logistic regression model by determining the probability a patient who experienced an event (in this case, a readmission) had a higher model risk than a patient who had not experienced the event. Calibration was evaluated by comparing predicted to observed probabilities for quintiles of risk. We calculated the categorical net reclassification improvement (NRI), which is the absolute net gain in correctly reclassified predictions of high (top 2 risk quintiles) and low risk (bottom 3 quintiles) for the 7-day readmission model compared to the 30-day model . To examine which factors were more (or less) weighted in the7-day readmission model, we evaluated the percent change in coefficients between the two models, using the 30-day model as reference. Odds ratios for variables were estimated from the coefficients of each logistic regression model.
Of 32,922 index hospitalizations among unique patients, 4.4% had a 7-day readmission and 12.7% had a 30-day readmission. Compared to those with 8-to-30-day readmissions, fewer patients with 7-day readmissions had one or more hospitalizations in the past year (43.3% vs. 47.7%, p = 0.01). On discharge, patients with 7-day readmissions had a higher proportion of one or greater vital sign instability (25.5% vs. 22.6%, p = 0.03) and sodium < 135 mEq/L (21.9% vs. 18.4%, p = 0.007) (Table 1). Our original 30-day model had modestly lower discrimination for predicting 7-day versus 30-day readmission (C-statistic of 0.66 vs. 0.69, p ≤ 0.001) (Fig. 1a). Our 7-day readmission model had similar discrimination as the original 30-day model for predicting 7-day readmissions (C-statistic of 0.66 vs. 0.66, p = 0.38) but improved calibration, particularly for the highest risk quintile (Fig. 1b). The 7-day model did not have better reclassification (NRI = 0.006, 95% CI: − 0.104 – 0.116).
When comparing strength of predictors between the two models, clinical characteristics at discharge, such as sodium and vital sign instability, were more strongly predictive of 7-day readmissions compared to 30-day readmissions (Table 2). Baseline characteristics (Medicaid and prior utilization), were less predictive of 7-day readmissions. Factors on admission and during the hospital stay also tended to be less predictive. Mean corpuscular volume > 100 fL/red cell (− 155%) and Albumin 2-3 g/dL (− 77%) had the greatest negative difference in coefficients from the 30-day model to the 7-day model (Fig. 2). The strongest statistically significant positive predictor of readmission was Clostridium difficile infection in the 30-day model and Albumin < 2 g/dL in the 7-day model. The strongest negative predictor in both models was discharge to hospice care. While all variables were statistically significant in the 30-day model, only 15 of the 24 variables were in the 7-day model.
While growing research [1,2,3,4,5,6,7] supports that 7-day readmissions are more preventable than 30-day readmissions, 30-day readmissions continue to dominate the readmission prediction space. Few studies explicitly have developed prediction models for 7-day readmissions [8, 12]. Herein, we provide empirical evidence that a previously validated, multi-condition 30-day EHR-based readmission risk prediction model can also be used to predict 7-day readmissions.
Performance for the original 30-day risk prediction model was not substantially different compared to a re-derived 7-day model. Reweighting coefficients for predictors led to slightly improved calibration, but risk stratification and reclassification of risk were similar. Therefore, until more robust 7-day specific readmission models are developed, this EHR-based 30-day model can be applied as an effective stopgap to also predict and target more preventable 7-day readmissions.
While overall model performance was similar, strength of predictors for 7-day versus 30-day readmission differed. Characteristics at discharge were more predictive of early 7-day readmissions, while baseline characteristics were less predictive. This is consistent with prior research suggesting that early readmissions are more likely to be related to clinical stability on discharge than 30-day readmissions [1,2,3,4,5,6]. Further research is needed to more conclusively compare predictive differences between those variables that were statistically significant in the 30-day model, but not in the 7-day model. Our study supports the inclusion of additional risk factors proximal to day of discharge such as the quality of transition of care planning (e.g. timely outpatient follow-up, medication reconciliation, and dispensing on discharge) to optimize performance of future 7-day readmission risk prediction models. Further optimizing risk prediction would enable hospitals to more efficiently target and reduce those readmissions that are potentially the most preventable.
Our study benefitted from the large, multicenter diverse cohort and high-quality ascertainment of readmissions beyond the index hospital. The use of rich, ubiquitous EHR data allows for real-time operationalization of the model wherein patients can be identified for intervention before they are discharged as is used in our hospital system . Furthermore, since we used the original cohort from which the 30-day readmission model was developed , we were uniquely positioned to isolate the ability of a 30-day readmission model to predict early 7-day readmissions by avoiding any differences in model performance stemming from changes in the study population itself. Study limitations include uncertain generalizability to other settings and use of data before federal penalties for hospital readmission were in effect.
A previously validated 30-day readmission model can be used as an effective stopgap for prediction of 7-day readmissions as model performance did not substantially change. However, strength of predictors differed between the 7-day and 30-day model informing future directions for predictive improvement, including greater focus on new risk factors proximal to day of discharge.
Availability of data and materials
The data that support the findings of this study are available from UT Southwestern Medical Center, but restrictions apply to the availability of these data, which were used under approval for the current study and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of UT Southwestern Medical Center.
Electronic health record
Net reclassification improvement
Auerbach AD, Kripalani S, Vasilevskis EE, Sehgal N, Lindenauer PK, Metlay JP, et al. Preventability and causes of readmissions in a national cohort of general medicine patients. JAMA Intern Med. 2016;176:484–93.
Chin DL, Bang H, Manickam RN, Romano PS. Rethinking thirty-day hospital readmissions: shorter intervals might be better indicators of quality of care. Health Aff. 2016;35:1867–75. https://doi.org/10.1377/hlthaff.2016.0205.
Graham KL, Wilker EH, Howell MD, Davis RB, Marcantonio ER. Differences between early and late readmissions among patients: a cohort study. Ann Intern Med. 2015;162:741. https://doi.org/10.7326/M14-2159.
Graham KL, Auerbach AD, Schnipper JL, Flanders SA, Kim CS, Robinson EJ, et al. Preventability of early versus late hospital readmissions in a national cohort of general medicine patients. Ann Intern Med. 2018;168:766. https://doi.org/10.7326/M17-1724.
Graham KL, Dike O, Doctoroff L, Jupiter M, Vanka A, Davis RB, et al. Preventability of early vs. late readmissions in an academic medical center. PLoS One. 2017;12:e0178718. https://doi.org/10.1371/journal.pone.0178718.
van Walraven C, Bennett C, Jennings A, Austin PC, Forster AJ. Proportion of hospital readmissions deemed avoidable: a systematic review. Can Med Assoc J. 2011;183:E391–402. https://doi.org/10.1503/cmaj.101860.
Koekkoek D, Bayley KB, Brown A, Rustvold DL. Hospitalists assess the causes of early hospital readmissions. J Hosp Med. 2011;6:383–8. https://doi.org/10.1002/jhm.909.
Artetxe A, Beristain A, Graña M. Predictive models for hospital readmission risk: a systematic review of methods. Comput Methods Prog Biomed. 2018;164:49–64. https://doi.org/10.1016/j.cmpb.2018.06.006.
Zhou H, Della PR, Roberts P, Goh L, Dhaliwal SS. Utility of models to predict 28-day or 30-day unplanned hospital readmissions: an updated systematic review. BMJ Open. 2016;6:e011060. https://doi.org/10.1136/bmjopen-2016-011060.
Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman M, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306:1688. https://doi.org/10.1001/jama.2011.1515.
Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc. 2017;24:198–208. https://doi.org/10.1093/jamia/ocw042.
Maali Y, Perez-Concha O, Coiera E, Roffe D, Day RO, Gallego B. Predicting 7-day, 30-day and 60-day all-cause unplanned readmission: a case study of a Sydney hospital. BMC Med Inform Decis Mak. 2018;18:1. https://doi.org/10.1186/s12911-017-0580-8.
Nguyen OK, Makam AN, Clark C, Zhang S, Xie B, Velasco F, et al. Predicting all-cause readmissions using electronic health record data from the entire hospitalization: model development and comparison. J Hosp Med. 2016;11:473–80.
Leening MJG, Vedder MM, Witteman JCM, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician’s guide. Ann Intern Med. 2014;160:122–31. https://doi.org/10.7326/M13-1522.
Amarasingham R, Patel PC, Toto K, Nelson LL, Swanson TS, Moore BJ, et al. Allocating scarce resources in real-time to reduce heart failure readmissions: a prospective, controlled study. BMJ Qual Saf. 2013;22:998–1005. https://doi.org/10.1136/bmjqs-2013-001901.
This work was supported by the Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (R24HS022418). Dr. Makam is funded by the National Institute on Aging (K23AG052603). Dr. Nguyen is funded by the National Heart, Lung, and Blood Institute (K23HL133441). The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Ethics approval and consent to participate
Approved through the Human Research Protection Program (HRPP) institutional review board (IRB) at UT Southwestern Medical Center under reference number STU 062010–146. Patient data were deidentified prior to data analysis and no administrative permission were required to access the raw data from patient medical records. Informed consent was waived by Human Research Protection Program (HRPP) institutional review board (IRB) at UT Southwestern Medical Center.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Saleh, S.N., Makam, A.N., Halm, E.A. et al. Can we predict early 7-day readmissions using a standard 30-day hospital readmission risk prediction model?. BMC Med Inform Decis Mak 20, 227 (2020). https://doi.org/10.1186/s12911-020-01248-1
- Hospital utilization
- Healthcare quality improvement
- Early readmissions
- Hospital medicine
- Care transitions
- Predictive model
- Clinical decision support