Fig. 1: Schematic illustration of the MMF-PH algorithmic framework for PH detection.

The figure outlines the MMF-PH framework, tailored for detecting PH from data collected across multiple healthcare centers. The top tier (a) shows the data collection phase, with datasets compiled from three centers treating patients with suspected PH who underwent RHC, the gold standard for PH evaluation. These datasets comprise tabular data (demographics, clinical features, medical histories, laboratory results), textual data (notes from CXRs, electrocardiography, and echocardiography), and imaging data from CXRs. The middle tier (b) delineates the data preprocessing and feature extraction phase. Here, tabular data is transformed by a transformer encoder, textual data by BERT, and imaging data by ViT, each extracting pertinent features to create embeddings—a standardized format for subsequent analysis. The final tier (c) presents the fusion and model construction phase. Embeddings from various sources are merged using a self-attention mechanism, central to the multimodal fusion process of the MMF-PH. This comprehensive integration leads to the formulation of the MMF-PH. AP anteroposterior, BERT Bidirectional Encoder Representations from Transformers, CXR chest X-ray, MMF-PH multimodal fusion model for pulmonary hypertension screening, MLP multilayer perceptron, RHC right heart catheterization, ViT Vision Transformer.