Skip to main content

Advertisement

SpringerLink
Log in

Machine learning and LACE index for predicting 30-day readmissions after heart failure hospitalization in elderly patients

  • IM - ORIGINAL
  • Published:
Internal and Emergency Medicine Aims and scope Submit manuscript

Abstract

Machine learning (ML) techniques may improve readmission prediction performance in heart failure (HF) patients. This study aimed to assess the ability of ML algorithms to predict unplanned all-cause 30-day readmissions in HF elderly patients, and to compare them with conventional LACE (Length of hospitalization, Acuity, Comorbidities, Emergency department visits) index. All patients aged ≥ 65 years discharged alive between 2010 and 2019 after a hospitalization for acute HF were included in this retrospective cohort study. We applied MICE (Multivariate Imputation via Chained Equations) method to obtain a balanced, fully valued dataset and LASSO (Least Absolute Shrinkage and Selection Operator) algorithm to get the most significant features. Training (80% of records) and test (20%) cohorts were randomly selected. Study population: 3079 patients, 394 (12.8%) presented at least one readmission within 30 days, and 2685 (87.2%) did not. In the test cohort AUCs (IC95%) of XGBoost, Ada Boost Classifier, Random forest, and Gradient Boosting, and LACE Index were: 0.803 (0.734–0.872), 0.782 (0.711–0.854), 0.776 (0.703–0.848), 0.786 (0.715–0.857), and 0.504 (0.414–0.594), respectively, for predicting readmissions. A SHAP analysis was performed to offer a breakdown of the ML variables associated with readmission. Positive and negative predicting values estimates of the different ML models and LACE index were also provided, for several values of readmission rate prevalence. Among elderly patients, the rate of all-cause unplanned 30-day readmissions after hospitalization due to an acute HF was high. ML models performed better than the conventional LACE index for predicting readmissions. ML models can be proposed as promising tools for the identification of subjects at high risk of hospitalization in this clinical setting, enabling care teams to target interventions for improving overall clinical outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Correale M, Paolillo S, Ruocco G, Palazzuoli A, Nodari S; from Italian Society of Cardiology (SIC) Working Group on Heart Failure: Comprehensive heart failure assessment: a challenge to modify the course of heart failure. Author's reply. Eur J Intern Med. 74:125-126. https://doi.org/10.1016/j.ejim.2020.01.020. Epub 2020 Feb 9. PMID: 32046942

  2. Ponikowski P, Voors AA, Anker SD, Bueno H, Cleland JGF, Coats AJS et al (2016) ESC guidelines for the diagnosis and treatment of acute and chronic heart failure. Eur Heart J 2015:ehw128

    Google Scholar 

  3. Bergethon K, Ju C, De Vore A, Hardy NC, Fonarow GC, Yancy CW, Heidenreich PA, Bhatt DL, Peterson ED, Hernandez AF (2016) Trends in 30-day readmission rates for patients hospitalized with heart failure. Circulation. 9:e002594. https://doi.org/10.1161/CIRCHEARTFAILURE.115.002594 (originally published June 14, 2016)

    Article  PubMed  Google Scholar 

  4. Ponikowski P, Anker SD, AlHabib KF, Cowie MR, Force TL, Hu S et al (2014) Heart failure: preventing disease and death worldwide. ESC Hear Fail 1:4–25

    Article  Google Scholar 

  5. Ambrosy AP, Fonarow GC, Butler J, Chioncel O, Greene SJ, Vaduganathan M et al (2014) The global health and economic burden of hospitalizations for heart failure: lessons learned from hospitalized heart failure registries. J Am Coll Cardiol Elsevier Inc 63:1123–1133. https://doi.org/10.1016/j.jacc.2013.11.053

    Article  Google Scholar 

  6. Ricciardi E, La Malfa G, Guglielmi G et al (2020) Characteristics of current heart failure patients admitted to internal medicine vs. cardiology hospital units: the VASCO study. Intern Emerg Med 15:1219–1229. https://doi.org/10.1007/s11739-020-02304-4

    Article  PubMed  Google Scholar 

  7. Belfiore A, Palmieri VO, Di Gennaro C et al (2020) Long-term management of chronic heart failure patients in internal medicine. Intern Emerg Med 15:49–58. https://doi.org/10.1007/s11739-019-02024-4

    Article  PubMed  Google Scholar 

  8. Piñeiro-Fernández J, Fernández-Rial Á, Suárez-Gil R et al (2021) Evaluation of a patient-centered integrated care program for individuals with frequent hospital readmissions and multimorbidity. Intern Emerg Med. https://doi.org/10.1007/s11739-021-02876-9

    Article  PubMed  Google Scholar 

  9. Senni M, Parrella P, De Maria R, Cottini C, Bohm M, Ponikowski P, Filippatos G, Tribouilloy C, Di Lenarda A, Oliva F, Pulignano G, Cicoira M, Nodari S, Porcu M, Cioffi G, Gabrielli D, Parodi O, Ferrazzi P, Gavazzi A (2011) Predicting heart failure outcome from cardiac and comorbid conditions: the 3C-HF score. Int J Cardiol. https://doi.org/10.1016/j.ijcard.2011.10.071

    Article  PubMed  Google Scholar 

  10. Averbuch T, Lee SF, Mamas MA, Oz UE, Perez R, Connolly SJ, Ko DT, Van Spall HGC (2021) Derivation and validation of a two-variable index to predict 30-day outcomes following heart failure hospitalization. ESC Heart Fail. https://doi.org/10.1002/ehf2.13324 (Epub ahead of print. PMID: 33932113)

    Article  PubMed  PubMed Central  Google Scholar 

  11. Van Walraven C, Dhalla IA, Bell Ch, Etchells E, Stiell IG (2010) Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ 6:51–557

    Google Scholar 

  12. Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C et al (2011) Risk prediction models for hospital readmission: a systematic review. JAMA 306(15):1688–1698

    Article  CAS  Google Scholar 

  13. Pocock SJ, Ariti CA, McMurray JJV, Maggioni A, Køber L, Squire IB et al (2013) Predicting survival in heart failure: a risk score based on 39 372 patients from 30 studies. Eur Heart J 34:1404–1413. https://doi.org/10.1093/eurheartj/ehs337 (PMID: 23095984)

    Article  PubMed  Google Scholar 

  14. Van Calster B, Verbakel JY, Christodoulou E, Steyerberg EW, Collins GS (2019) Statistics versus machine learning: definitions are interesting (but understanding, methodology, and reporting are more important). J Clin Epidemiol 116:137–138. https://doi.org/10.1016/j.jclinepi.2019.08.002 (Epub 2019 Aug 16 PMID: 31425736)

    Article  PubMed  Google Scholar 

  15. Zhang Z, Zhang H, Khanal MK (2017) Development of scoring system for risk stratification in clinical medicine: a step-by-step tutorial. Ann Transl Med 5(21):436. https://doi.org/10.21037/atm.2017.08.22.PMID:29201888;PMCID:PMC5690964

    Article  PubMed  PubMed Central  Google Scholar 

  16. Sun GW, Shook TL, Kay GL (1996) Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. J Clin Epidemiol 49(8):907–916. https://doi.org/10.1016/0895-4356(96)00025-x (PMID: 8699212)

    Article  CAS  PubMed  Google Scholar 

  17. Bagley SC, White H, Golomb BA (2001) Logistic regression in the medical literature: standards for use and reporting, with particular attention to one medical domain. J Clin Epidemiol 54:979–985 (PMID: 11576808)

    Article  CAS  Google Scholar 

  18. Levy WC, Anand IS (2014) Heart failure risk prediction models: what have we learned? JACC Heart Fail 2:437–439 ([PubMed: 25194289])

    Article  Google Scholar 

  19. Grant L, Joo P, Nemnom MJ et al (2021) Machine learning versus traditional methods for the development of risk stratification scores: a case study using original Canadian Syncope Risk Score data. Intern Emerg Med. https://doi.org/10.1007/s11739-021-02873-y

    Article  PubMed  Google Scholar 

  20. Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B (2019) A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol 110:12–22. https://doi.org/10.1016/j.jclinepi.2019.02.004 (Epub 2019 Feb 11 PMID: 30763612)

    Article  PubMed  Google Scholar 

  21. Eapen ZJ, Liang L, Fonarow GC, Heidenreich PA, Curtis LH, Peterson ED, Hernandez AF (2013) Validated, electronic health record deployable prediction models for assessing patient risk of 30-day rehospitalization and mortality in older heart failure patients. JACC Heart Fail 1:245–251. https://doi.org/10.1016/j.jchf.2013.01.008

    Article  PubMed  Google Scholar 

  22. Dedalus spa. Tabula Clinica. http://www.dedalus.eu (Accessed December 2019).

  23. van Buuren S, Boshuizen HC, Knook DL (1999) Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med 18:681–694

    Article  Google Scholar 

  24. Tibshirani R (1996) Regression Shrinkage and Selection via the lasso. J R Stat Soc Ser B (Methodol) Wiley 58 (1): 267–88 (JSTOR 2346178)

  25. Gruneir A, Dhalla IA, van Walraven C, Fischer HD, Camacho X, Rochon PA, Anderson GM (2011) Unplanned readmissions after hospital discharge among patients identified as being at high risk for readmission using a validated predictive algorithm. Open Med 5(2):e104–e111 (Epub 2011 May 31)

    PubMed  PubMed Central  Google Scholar 

  26. Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6:52138–52160

    Article  Google Scholar 

  27. Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. Adv Neural Inf Process Syst 30:4765–4774

    Google Scholar 

  28. Hayden SR, Brown MD (1999) Likelihood ratio: a powerful tool for incorporating the results of a diagnostic test into clinical decisionmaking. Ann Emerg Med 33(5):575–580. https://doi.org/10.1016/s0196-0644(99)70346-x (PMID: 10216335)

    Article  CAS  PubMed  Google Scholar 

  29. Andrès E, Talha S, Hajjam M, Hajjam J, Ervé S, Hajjam A (2018) Experimentation of 2.0 telemedicine in elderly patients with chronic heart failure: a study prospective in 175 patients. Eur J Intern Med. 51:e11–e12. https://doi.org/10.1016/j.ejim.2018.02.022 (Epub 2018 Mar 7. PMID: 29525507)

    Article  PubMed  Google Scholar 

  30. Golas SB, Shibahara T, Agboola S, Otaki H, Sato J, Nakae T, Hisamitsu T, Kojima G, Felsted J, Kakarmath S, Kvedar J, Jethwani K (2018) A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak 18(1):44. https://doi.org/10.1186/s12911-018-0620-z

    Article  PubMed  PubMed Central  Google Scholar 

  31. Mahajan SM, Mahajan AS, King R, Negahban S (2018) Predicting risk of 30-day readmissions using two emerging machine learning methods. Stud Health Technol Inform 250:250–255

    PubMed  Google Scholar 

  32. Mortazavi BJ, Downing NS, Bucholz EM et al (2016) Analysis of machine learning techniques for heart failure readmissions. Circ Cardiovasc Qual Outcomes 9:629–640

    Article  Google Scholar 

  33. Wang H, Robinson RD, Johnson C, Zenarosa NR, Jayswal RD, Keithley J, Delaney KA (2014) Using the LACE index to predict hospital readmissions in congestive heart failure patients. BMC Cardiovasc Disord 14:97. https://doi.org/10.1186/1471-2261-14-97

    Article  PubMed  PubMed Central  Google Scholar 

  34. Wongvibulsin S, Wu KC, Zeger SL (2020) Improving clinical translation of machine learning approaches through clinician-tailored visual displays of black box algorithms: development and validation. JMIR Med Inform 8(6):e15791. https://doi.org/10.2196/15791.PMID:32515746;PMCID:PMC7312245

    Article  PubMed  PubMed Central  Google Scholar 

  35. Kajimoto K, Sato N, Keida T, Sakata Y, Asai K, Mizuno M, Takano T, Investigators of the Acute Decompensated Heart Failure Syndromes (ATTEND) registry (2014) Low admission heart rate is a marker rather than a mediator of increased in-hospital mortality for patients with acute heart failure syndromes in sinus rhythm. Int J Cardiol. 171(1):98–100. https://doi.org/10.1016/j.ijcard.2013.11.087 (Epub 2013 Dec 4. PMID: 24342405)

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Internal Medicine, Medical Department, Vimercate Hospital, Azienda Socio Sanitaria Territoriale (ASST) della Brianza, Via Santi Cosma e Damiano 10, 20871, Vimercate, MB, Italy

    Hernan Polo Friz & Laura Primitz

  2. Almaviva Digitaltec S.R.L, Naples, Italy

    Valentina Esposito

  3. Department of Biomedical and Clinical Sciences “L. Sacco”, University of Milan, Milan, Italy

    Giuseppe Marano & Patrizia Boracchi

  4. Almaviva S.P.a, Rome, Italy

    Alice Bovio

  5. Information Technology Department, ASST della Brianza, Vimercate, Italy

    Giovanni Delgrossi

  6. Internal Medicine, Medical Department, Desio Hospital, ASST della Brianza, Desio, Italy

    Michele Bombelli

  7. Director for Health and Social Care, ASST della Brianza, Vimercate, Italy

    Guido Grignaffini

  8. Medical Director, ASST della Brianza, Vimercate, Italy

    Giovanni Monza

Authors
  1. Hernan Polo Friz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Valentina Esposito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giuseppe Marano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Laura Primitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alice Bovio
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Giovanni Delgrossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michele Bombelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Guido Grignaffini
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Giovanni Monza
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Patrizia Boracchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HPF: Conceptualization, methodology, formal analysis, investigation, data curation, writing review and editing, supervision, project administration. VE: Conceptualization, ML models development, data curation, writing review and editing. GM: Methodology, formal statistical analysis, data curation, writing review and editing. LP: Investigation, data curation, interpretation of results, writing review and editing. AV: ML models development, data curation, writing review and editing. GD: Conceptualization, investigation, interpretation of results, writing review and editing. MB: Conceptualization, investigation, interpretation of results, writing review and editing. GG: Conceptualization, methodology, interpretation of results, writing review and editing. GM: Conceptualization, methodology, interpretation of results, writing review and editing. PB: Conceptualization, methodology, formal statistical analysis, data curation, writing review and editing, supervision.

Corresponding author

Correspondence to Hernan Polo Friz.

Ethics declarations

Conflict of interest

The author(s) declare that they have no conflict of interest.

Human and animal rights statement and Informed consent

Ethical approval was required to the Institutional Research Ethics Committee (Comitato Etico Brianza), and informed consent was waived given the retrospective non-interventional, observational design.

Writing assistance

None to declare. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The work has not been published previously and it is not under consideration for publication elsewhere.

Authorship

Authors declare that all have made substantial contributions to the conceptualization and design of the work, the acquisition, analysis, and the interpretation of data. All authors revised critically the draft adding important intellectual content. All authors approve the final version to be published.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Polo Friz, H., Esposito, V., Marano, G. et al. Machine learning and LACE index for predicting 30-day readmissions after heart failure hospitalization in elderly patients. Intern Emerg Med 17, 1727–1737 (2022). https://doi.org/10.1007/s11739-022-02996-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11739-022-02996-w

Keywords

Search

Navigation