CARDIAC-FM: A Generalizable Multimodal Foundation Model Integrating ECG and Cardiac MRI

Atrial fibrillation and heart failure impose substantial health burdens worldwide, yet accurate and generalizable risk prediction remains challenging. Here we developed CARDIAC-FM, a multimodal foundation model that integrates 12-lead electrocardiography (ECG) and cardiac magnetic resonance imaging (cardiac MRI) through self-supervised representation learning and cross-modal contrastive alignment. CARDIAC-FM introduces a self-supervised spatiotemporal, multi-view masked autoencoder for cardiac MRI. Developed on 57,609 paired ECG-cardiac MRI samples from UK Biobank, CARDIAC-FM improved prediction of incident atrial fibrillation and heart failure over contemporary ECG AI models and generalized zero-shot to two external cohorts, the Cardiovascular Health Study and the Multi-Ethnic Study of Atherosclerosis. Combining its ECG representation with established clinical risk scores further improved discrimination across cohorts, supporting complementary prognostic information from ECG and traditional risk factors. Beyond atrial fibrillation and heart failure, the learned representation transferred to continuous cardiac MRI phenotype prediction, time-to-event modelling and prediction of additional cardiovascular outcomes with limited fine-tuning, including myocardial infarction, ischaemic stroke, cardiovascular death and all-cause mortality. Although pre-trained with paired ECG and cardiac MRI, the model can be deployed using ECG alone, with additional predictive gains when cardiac MRI is available. These findings demonstrate the promise of multimodal self-supervised learning for generalizable cardiovascular risk prediction.