OT-CGSF: Optimal Transport and CNN-Guided Score-Based Fusion for Multimodal Medical Image Integration

Effective fusion of multimodal medical images is crucial for accurate and precise disease diagnosis. This study introduces a novel framework, Optimal Transport and CNN-Guided Score-Based Fusion (OT-CGSF), which effectively align and merge information from diverse imaging modalities. The proposed method begins by aligning intensity distributions using Optimal Transport (OT), ensuring coherent fusion. A pre trained ResNet Convolutional Neural Network (CNN) then extracts informative feature maps, which are refined through guided filtering to enhance spatial consistency. Moreover, a score-based mechanism dynamically adjusts spatial weights, optimizing the fusion outcomes. Additionally, weights and scores are refined by Guided filter to enhances the adaptive combination of images to improve image quality. Furthermore, Image fusion is carried on using softmax averaging method which combines images by normalizing their weights to enhance image clarity and to preserve key features. Authors have also performed comparative analysis with state-of-the-art methods which shows that OT-CGSF excels in preserving information and suppressing artifacts. The method’s effectiveness is further validated in the fusion of other databases of color multi-focus and infrared (IV) images, demonstrating its effective applicability in diverse scenario. OT-CGSF represents an innovative and versatile fusion paradigm that significantly enhances diagnostic accuracy across various medical imaging scenarios.