Fusing Echocardiography Images and Medical Records for Continuous Patient Stratification
Deep learning enables automatic and robust extraction of cardiac function descriptors from echocardiographic sequences, such as ejection fraction (EF) or strain. These descriptors provide fine-grained information that physicians consider, in conjunction with more global variables from the clinical r...
| Publié dans: | IEEE transactions on ultrasonics, ferroelectrics, and frequency control. - 1986. - 72(2025), 10 vom: 20. Sept., Seite 1388-1400 |
|---|---|
| Auteur principal: | |
| Autres auteurs: | , , , , , |
| Format: | Article en ligne |
| Langue: | English |
| Publié: |
2025
|
| Accès à la collection: | IEEE transactions on ultrasonics, ferroelectrics, and frequency control |
| Sujets: | Journal Article |
| Résumé: | Deep learning enables automatic and robust extraction of cardiac function descriptors from echocardiographic sequences, such as ejection fraction (EF) or strain. These descriptors provide fine-grained information that physicians consider, in conjunction with more global variables from the clinical record, to assess patients' condition. Drawing on novel Transformer models applied to tabular data, we propose a method that considers all descriptors extracted from medical records and echocardiograms to learn the representation of a cardiovascular pathology with a difficult-to-characterize continuum, namely hypertension. Our method first projects each variable into its own representation space using modality-specific approaches. These standardized representations of multimodal data are then fed to a Transformer encoder, which learns to merge them into a comprehensive representation of the patient through the task of predicting a clinical rating. This stratification task is formulated as an ordinal classification to enforce a pathological continuum in the representation space. We observe the major trends along this continuum on a cohort of 239 hypertensive patients, providing unprecedented details in the description of hypertension's impact on various cardiac function descriptors. Our analysis shows that: 1) the XTab foundation model's architecture allows to reach high performance (96.8% AUROC) even with limited data (less than 200 training samples); 2) stratification across the population is reproducible between trainings [within 5.7% of mean absolute error (MAE)]; and 3) patterns emerge in descriptors, some of which align with established physiological knowledge about hypertension, while others could pave the way for a more comprehensive understanding of this pathology. The code is available at https://github.com/creatis-myriad/didactic |
|---|---|
| Description: | Date Completed 29.09.2025 Date Revised 29.09.2025 published: Print Citation Status MEDLINE |
| ISSN: | 1525-8955 |
| DOI: | 10.1109/TUFFC.2025.3600902 |