Development and external validation of a deep learning algorithm for prognostication of cardiovascular outcomes

In Jeong Cho, Ji Min Sung, Hyeon Chang Kim, Sang Eun Lee, Myeong Hun Chae, Maryam Kavousi, Oscar L. Rueda-Ochoa, M. Arfan Ikram, Oscar H. Franco, James K. Min, Hyuk Jae Chang

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


Background and Objectives: We aim to explore the additional discriminative accuracy of a deep learning (DL) algorithm using repeated-measures data for identifying people at high risk for cardiovascular disease (CVD), compared to Cox hazard regression. Methods: Two CVD prediction models were developed from National Health Insurance Service-Health Screening Cohort (NHIS-HEALS): A Cox regression model and a DL model. Performance of each model was assessed in the internal and 2 external validation cohorts in Koreans (National Health Insurance Service-National Sample Cohort; NHIS-NSC) and in Europeans (Rotterdam Study). A total of 412,030 adults in the NHIS-HEALS; 178,875 adults in the NHIS-NSC; and the 4,296 adults in Rotterdam Study were included. Results: Mean ages was 52 years (46% women) and there were 25,777 events (6.3%) in NHIS-HEALS during the follow-up. In internal validation, the DL approach demonstrated a C-statistic of 0.896 (95% confidence interval, 0.886-0.907) in men and 0.921 (0.908-0.934) in women and improved reclassification compared with Cox regression (net reclassification index [NRI], 24.8% in men, 29.0% in women). In external validation with NHIS-NSC, DL demonstrated a C-statistic of 0.868 (0.860-0.876) in men and 0.889 (0.876-0.898) in women, and improved reclassification compared with Cox regression (NRI, 24.9% in men, 26.2% in women). In external validation applied to the Rotterdam Study, DL demonstrated a C-statistic of 0.860 (0.824-0.897) in men and 0.867 (0.830-0.903) in women, and improved reclassification compared with Cox regression (NRI, 36.9% in men, 31.8% in women). Conclusions: A DL algorithm exhibited greater discriminative accuracy than Cox model approaches.

Original languageEnglish
Pages (from-to)72-84
Number of pages13
JournalKorean Circulation Journal
Issue number1
Publication statusPublished - 2020

Bibliographical note

Funding Information:
This research was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korean government (MSIT) (No. 2017-0-00255, Autonomous Digital Companion Development and No.2018-0-00861, Intelligent SW Technology Development for Medical Data Analysis).

Funding Information:
The Rotterdam Study is funded by Erasmus MC and Erasmus University, Rotterdam, The Netherlands; The Netherlands Organisation for Scientific Research (NWO); The Netherlands Organisation for the Health Research and Development (ZonMw); The Research Institute for Diseases in the Elderly (RIDE); The Ministry of Education, Culture and Science; the Ministry for Health, Welfare and Sports; The European Commission (DG XII); and The Municipality of Rotterdam. M. Kavousi is supported by the NWO VENI grant (VENI, 91616079). Oscar L. Rueda is supported by a scholarship by COLCIENCIAS and Universidad Industrial de Santander from Colombia. O.H. Franco works in ErasmusAGE, a center for aging research across the life course funded by Nestlé Nutrition (Nestec Ltd.); Metagenics Inc.; and AXA. The funding sources had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review or approval of the manuscript; and decision to submit the manuscript for publication.

Publisher Copyright:
© 2020. The Korean Society of Cardiology.

All Science Journal Classification (ASJC) codes

  • Internal Medicine
  • Cardiology and Cardiovascular Medicine


Dive into the research topics of 'Development and external validation of a deep learning algorithm for prognostication of cardiovascular outcomes'. Together they form a unique fingerprint.

Cite this