Skip to main content

Can we predict early 7-day readmissions using a standard 30-day hospital readmission risk prediction model?

Abstract

Background

Despite focus on preventing 30-day readmissions, early readmissions (within 7 days of discharge) may be more preventable than later readmissions (8–30 days). We assessed how well a previously validated 30-day EHR-based readmission prediction model predicts 7-day readmissions and compared differences in strength of predictors.

Methods

We conducted an observational study on adult hospitalizations from 6 diverse hospitals in North Texas using a 50–50 split-sample derivation and validation approach. We re-derived model coefficients for the same predictors as in the original 30-day model to optimize prediction of 7-day readmissions. We then compared the discrimination and calibration of the 7-day model to the 30-day model to assess model performance. To examine the changes in the point estimates between the two models, we evaluated the percent changes in coefficients.

Results

Of 32,922 index hospitalizations among unique patients, 4.4% had a 7-day admission and 12.7% had a 30-day readmission. Our original 30-day model had modestly lower discrimination for predicting 7-day vs. any 30-day readmission (C-statistic of 0.66 vs. 0.69, p ≤ 0.001). Our re-derived 7-day model had similar discrimination (C-statistic of 0.66, p = 0.38), but improved calibration. For the re-derived 7-day model, discharge day factors were more predictive of early readmissions, while baseline characteristics were less predictive.

Conclusion

A previously validated 30-day readmission model can also be used as a stopgap to predict 7-day readmissions as model performance did not substantially change. However, strength of predictors differed between the 7-day and 30-day model; characteristics at discharge were more predictive of 7-day readmissions, while baseline characteristics were less predictive. Improvements in predicting early 7-day readmissions will likely require new risk factors proximal to day of discharge.

Peer Review reports

Background

Despite intense focus on preventing 30-day readmissions, early readmissions within the first 7 days of hospital discharge may be more preventable than later readmissions (8–30 days post-discharge) [1,2,3,4,5,6,7]. Early readmissions are more closely related to potential gaps in care during the index hospitalization [4] or reflect premature discharge [7]. Identifying patients at risk for early, rather than later readmissions may be a more effective strategy to tailor resource-intensive transitional care interventions to prevent readmissions. However, current risk prediction models often only identify patients at risk for 30-day readmission [8,9,10] and often fail to use electronic health record (EHR) data effectively to allow for real-time operationalization of the model [10, 11]. There is a paucity of research developing prediction models for adult 7-day readmissions [8, 12], which may be due to federal financial penalties tied to 30-day readmissions. Yet, to our knowledge, no study has investigated if a 30-day model can be reapplied effectively to predict the important subset of 7-day readmissions. Therefore, we assessed how well a validated 30-day EHR-based readmission risk prediction model [13] would predict early 7-day readmissions, and whether there were differences in the strength of predictors for 7-day versus 30-day readmissions.

Methods

We conducted an observational cohort study of consecutive hospitalizations by adults ≥18 years from November 2009 to October 2010 using electronic health record (EHR) data from 6 diverse hospitals in north Texas, including safety-net, academic, and community sites. We included index hospitalizations of patients who were alive 30 days post-discharge. Patients who died in the hospital, were transferred to another hospital, or left against medical advice were excluded. The primary outcome was all-cause non-elective 7-day hospital readmissions within a 100-mile radius of Dallas, Texas (includes 75 acute care hospitals), which were retrieved from an all-payer regional hospitalization database. The previously validated 30-day readmission multivariate logistic regression model was developed using a 50–50 derivation-validation split. Multiple groups of candidate predictors including socioeconomic, admission day, hospital stay, and discharge day variables were included in development of the model. All variables were available in the EHR for all participating hospitals and were plausible based on previous literature and clinical expertise. Further details about model development have been previously published [13].

Using the same derivation cohort, we ran a multivariate logistic regression to re-derive model coefficients for the same predictors from our validated 30-day readmission model [13] (also developed from the same cohort) to optimize prediction of 7-day readmissions. Using the same cohort allows for direct comparison of model performance and changes in coefficient direction and magnitude between models. We used the validation cohort to compare the discrimination and calibration of our 7-day readmission model with our original 30-day model to predict 7-day readmissions. Discrimination was assessed using the C statistic, which measures the goodness of fit of a logistic regression model by determining the probability a patient who experienced an event (in this case, a readmission) had a higher model risk than a patient who had not experienced the event. Calibration was evaluated by comparing predicted to observed probabilities for quintiles of risk. We calculated the categorical net reclassification improvement (NRI), which is the absolute net gain in correctly reclassified predictions of high (top 2 risk quintiles) and low risk (bottom 3 quintiles) for the 7-day readmission model compared to the 30-day model [14]. To examine which factors were more (or less) weighted in the7-day readmission model, we evaluated the percent change in coefficients between the two models, using the 30-day model as reference. Odds ratios for variables were estimated from the coefficients of each logistic regression model.

Results

Of 32,922 index hospitalizations among unique patients, 4.4% had a 7-day readmission and 12.7% had a 30-day readmission. Compared to those with 8-to-30-day readmissions, fewer patients with 7-day readmissions had one or more hospitalizations in the past year (43.3% vs. 47.7%, p = 0.01). On discharge, patients with 7-day readmissions had a higher proportion of one or greater vital sign instability (25.5% vs. 22.6%, p = 0.03) and sodium < 135 mEq/L (21.9% vs. 18.4%, p = 0.007) (Table 1). Our original 30-day model had modestly lower discrimination for predicting 7-day versus 30-day readmission (C-statistic of 0.66 vs. 0.69, p ≤ 0.001) (Fig. 1a). Our 7-day readmission model had similar discrimination as the original 30-day model for predicting 7-day readmissions (C-statistic of 0.66 vs. 0.66, p = 0.38) but improved calibration, particularly for the highest risk quintile (Fig. 1b). The 7-day model did not have better reclassification (NRI = 0.006, 95% CI: − 0.104 – 0.116).

Table 1 Descriptive characteristics of 8-to-30-day vs. 7-day readmissions
Fig. 1
figure1

Model Performance of the 7-day versus 30-day Readmission Models. a Receiver operating characteristic (ROC) curves. The original 30-day model had modestly lower discrimination for predicting 7-day versus 30-day readmission (C-statistic of 0.66 vs. 0.69, p ≤ 0.001). Our re-derived 7-day readmission model had similar discrimination as the original 30-day model for predicting 7-day readmissions (C-statistic of 0.66 vs. 0.66, p = 0.38). b Calibration. To predict 7-day readmissions, the new 7-day prediction model had better calibration than the original 30-day prediction model across all quintiles of risk, but risk stratification was similar

When comparing strength of predictors between the two models, clinical characteristics at discharge, such as sodium and vital sign instability, were more strongly predictive of 7-day readmissions compared to 30-day readmissions (Table 2). Baseline characteristics (Medicaid and prior utilization), were less predictive of 7-day readmissions. Factors on admission and during the hospital stay also tended to be less predictive. Mean corpuscular volume > 100 fL/red cell (− 155%) and Albumin 2-3 g/dL (− 77%) had the greatest negative difference in coefficients from the 30-day model to the 7-day model (Fig. 2). The strongest statistically significant positive predictor of readmission was Clostridium difficile infection in the 30-day model and Albumin < 2 g/dL in the 7-day model. The strongest negative predictor in both models was discharge to hospice care. While all variables were statistically significant in the 30-day model, only 15 of the 24 variables were in the 7-day model.

Table 2 Comparing strength of predictors of 30-day vs. 7-day readmissionsa
Fig. 2
figure2

Change in Strength of Predictors. The percent change in β coefficients between the original 30-day model and the re-derived 7-day model is shown for each predictor included in the model. Predictors are grouped according to the timing of their availability, including baseline characteristics prior to the index hospitalization (red), factors on hospital admission (light blue), factors during hospital stay (gold) and factors on discharge day (dark blue). Values to the right of the vertical dashed line at 0, shown in striped color, indicate factors that are more predictive of early readmission. Values to the left of the dashed line showed in solid color, indicate factors that are less predictive. Variables that are still statistically significant in the 7-day model are denoted with an asterisk (*)

Discussion

While growing research [1,2,3,4,5,6,7] supports that 7-day readmissions are more preventable than 30-day readmissions, 30-day readmissions continue to dominate the readmission prediction space. Few studies explicitly have developed prediction models for 7-day readmissions [8, 12]. Herein, we provide empirical evidence that a previously validated, multi-condition 30-day EHR-based readmission risk prediction model can also be used to predict 7-day readmissions.

Performance for the original 30-day risk prediction model was not substantially different compared to a re-derived 7-day model. Reweighting coefficients for predictors led to slightly improved calibration, but risk stratification and reclassification of risk were similar. Therefore, until more robust 7-day specific readmission models are developed, this EHR-based 30-day model can be applied as an effective stopgap to also predict and target more preventable 7-day readmissions.

While overall model performance was similar, strength of predictors for 7-day versus 30-day readmission differed. Characteristics at discharge were more predictive of early 7-day readmissions, while baseline characteristics were less predictive. This is consistent with prior research suggesting that early readmissions are more likely to be related to clinical stability on discharge than 30-day readmissions [1,2,3,4,5,6]. Further research is needed to more conclusively compare predictive differences between those variables that were statistically significant in the 30-day model, but not in the 7-day model. Our study supports the inclusion of additional risk factors proximal to day of discharge such as the quality of transition of care planning (e.g. timely outpatient follow-up, medication reconciliation, and dispensing on discharge) to optimize performance of future 7-day readmission risk prediction models. Further optimizing risk prediction would enable hospitals to more efficiently target and reduce those readmissions that are potentially the most preventable.

Our study benefitted from the large, multicenter diverse cohort and high-quality ascertainment of readmissions beyond the index hospital. The use of rich, ubiquitous EHR data allows for real-time operationalization of the model wherein patients can be identified for intervention before they are discharged as is used in our hospital system [15]. Furthermore, since we used the original cohort from which the 30-day readmission model was developed [13], we were uniquely positioned to isolate the ability of a 30-day readmission model to predict early 7-day readmissions by avoiding any differences in model performance stemming from changes in the study population itself. Study limitations include uncertain generalizability to other settings and use of data before federal penalties for hospital readmission were in effect.

Conclusions

A previously validated 30-day readmission model can be used as an effective stopgap for prediction of 7-day readmissions as model performance did not substantially change. However, strength of predictors differed between the 7-day and 30-day model informing future directions for predictive improvement, including greater focus on new risk factors proximal to day of discharge.

Availability of data and materials

The data that support the findings of this study are available from UT Southwestern Medical Center, but restrictions apply to the availability of these data, which were used under approval for the current study and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of UT Southwestern Medical Center.

Abbreviations

EHR:

Electronic health record

NRI:

Net reclassification improvement

ED:

Emergency department

References

  1. 1.

    Auerbach AD, Kripalani S, Vasilevskis EE, Sehgal N, Lindenauer PK, Metlay JP, et al. Preventability and causes of readmissions in a national cohort of general medicine patients. JAMA Intern Med. 2016;176:484–93.

    Article  Google Scholar 

  2. 2.

    Chin DL, Bang H, Manickam RN, Romano PS. Rethinking thirty-day hospital readmissions: shorter intervals might be better indicators of quality of care. Health Aff. 2016;35:1867–75. https://doi.org/10.1377/hlthaff.2016.0205.

    Article  Google Scholar 

  3. 3.

    Graham KL, Wilker EH, Howell MD, Davis RB, Marcantonio ER. Differences between early and late readmissions among patients: a cohort study. Ann Intern Med. 2015;162:741. https://doi.org/10.7326/M14-2159.

    Article  PubMed  PubMed Central  Google Scholar 

  4. 4.

    Graham KL, Auerbach AD, Schnipper JL, Flanders SA, Kim CS, Robinson EJ, et al. Preventability of early versus late hospital readmissions in a national cohort of general medicine patients. Ann Intern Med. 2018;168:766. https://doi.org/10.7326/M17-1724.

    Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    Graham KL, Dike O, Doctoroff L, Jupiter M, Vanka A, Davis RB, et al. Preventability of early vs. late readmissions in an academic medical center. PLoS One. 2017;12:e0178718. https://doi.org/10.1371/journal.pone.0178718.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    van Walraven C, Bennett C, Jennings A, Austin PC, Forster AJ. Proportion of hospital readmissions deemed avoidable: a systematic review. Can Med Assoc J. 2011;183:E391–402. https://doi.org/10.1503/cmaj.101860.

    Article  Google Scholar 

  7. 7.

    Koekkoek D, Bayley KB, Brown A, Rustvold DL. Hospitalists assess the causes of early hospital readmissions. J Hosp Med. 2011;6:383–8. https://doi.org/10.1002/jhm.909.

    Article  PubMed  Google Scholar 

  8. 8.

    Artetxe A, Beristain A, Graña M. Predictive models for hospital readmission risk: a systematic review of methods. Comput Methods Prog Biomed. 2018;164:49–64. https://doi.org/10.1016/j.cmpb.2018.06.006.

    Article  Google Scholar 

  9. 9.

    Zhou H, Della PR, Roberts P, Goh L, Dhaliwal SS. Utility of models to predict 28-day or 30-day unplanned hospital readmissions: an updated systematic review. BMJ Open. 2016;6:e011060. https://doi.org/10.1136/bmjopen-2016-011060.

    Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman M, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306:1688. https://doi.org/10.1001/jama.2011.1515.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  11. 11.

    Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc. 2017;24:198–208. https://doi.org/10.1093/jamia/ocw042.

    Article  PubMed  Google Scholar 

  12. 12.

    Maali Y, Perez-Concha O, Coiera E, Roffe D, Day RO, Gallego B. Predicting 7-day, 30-day and 60-day all-cause unplanned readmission: a case study of a Sydney hospital. BMC Med Inform Decis Mak. 2018;18:1. https://doi.org/10.1186/s12911-017-0580-8.

    Article  PubMed  PubMed Central  Google Scholar 

  13. 13.

    Nguyen OK, Makam AN, Clark C, Zhang S, Xie B, Velasco F, et al. Predicting all-cause readmissions using electronic health record data from the entire hospitalization: model development and comparison. J Hosp Med. 2016;11:473–80.

    Article  Google Scholar 

  14. 14.

    Leening MJG, Vedder MM, Witteman JCM, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician’s guide. Ann Intern Med. 2014;160:122–31. https://doi.org/10.7326/M13-1522.

    Article  PubMed  Google Scholar 

  15. 15.

    Amarasingham R, Patel PC, Toto K, Nelson LL, Swanson TS, Moore BJ, et al. Allocating scarce resources in real-time to reduce heart failure readmissions: a prospective, controlled study. BMJ Qual Saf. 2013;22:998–1005. https://doi.org/10.1136/bmjqs-2013-001901.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This work was supported by the Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (R24HS022418). Dr. Makam is funded by the National Institute on Aging (K23AG052603). Dr. Nguyen is funded by the National Heart, Lung, and Blood Institute (K23HL133441). The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Author information

Affiliations

Authors

Contributions

Study concept and design: SS, ANM, OKN; Data acquisition: EAH; Analysis: SS; Interpretation of data: all authors; Manuscript preparation: SS, ANM, OKN. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sameh N. Saleh.

Ethics declarations

Ethics approval and consent to participate

Approved through the Human Research Protection Program (HRPP) institutional review board (IRB) at UT Southwestern Medical Center under reference number STU 062010–146. Patient data were deidentified prior to data analysis and no administrative permission were required to access the raw data from patient medical records. Informed consent was waived by Human Research Protection Program (HRPP) institutional review board (IRB) at UT Southwestern Medical Center.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Saleh, S.N., Makam, A.N., Halm, E.A. et al. Can we predict early 7-day readmissions using a standard 30-day hospital readmission risk prediction model?. BMC Med Inform Decis Mak 20, 227 (2020). https://doi.org/10.1186/s12911-020-01248-1

Download citation

Keywords

  • Hospital utilization
  • Healthcare quality improvement
  • Early readmissions
  • Hospital medicine
  • Care transitions
  • Predictive model
  • Clinical decision support