Abstract
In medical science, there is a challenge to find out critical information from the medical images by low vision disability medical experts. As a solution, we can enhance the medical images by fusing different modality images viz., CT-MRI which can be more informative. This article presents a new multi-modal medical image fusion architecture in non-subsampled contourlet transform (NSCT) domain which is shift-invariant over noisy medical images. Initially noise from medical images is reduced using a convolution neural network (CNN) approach. Furthermore, NSCT is applied in denoised source multi-modal images to obtain approximation and detailed parts. In approximation parts of both input images, the fusion operation is performed using Direction Total Variation enabled linear spectral clustering. Simlarly in detailed parts of both input images fusion operation is performed using sum modified laplacian (SML) approaches. By performing inverse operation on both modified approximation and detailed parts, final fused image is obtained. From qualitative and quantitative result analysis, it can be concluded that the proposed method is an essential means of ensuring that multi-modality images provide more reliable analytical results to analyze experimental outcomes and comparative research.
Data availability
The dataset analyzed during the current study is available from the corresponding author (i.e. Manoj Diwakar) on reasonable request.
References
Singh, R. & Khare, A. Redundant discrete wavelet transform based medical image fusion. In Advances in Signal Processing and Intelligent Recognition Systems 505–515 (Springer, 2014). https://doi.org/10.1007/978-3-319-04960-1_44.
Bhatnagar, G., Wu, Q. J. & Liu, Z. A new contrast based multi-modal medical image fusion framework. Neurocomputing 157, 143–152. https://doi.org/10.1016/j.neucom.2015.01.025 (2015).
Bindu, C. H. & Prasad, K. S. Automatic region segmentation and variance based multi-modal medical image fusion. In Cognitive Science and Health Bioinformatics 57–63 (Springer, 2018).
Pohl, C. & Nazirun, N. N. Tamin SS multimodal medical image fusion in cardiovascular applications. In Medical Imaging Technology 91–109 (Springer, 2015). https://doi.org/10.1007/978-981-287-540-2_4.
Wu, D., Yang, A., Zhu, L. & Zhang, C. Survey of multi-sensor image fusion. In: In International Conference on Life System Modeling and Simulation and International Conference on Intelligent Computing for Sustainable Energy and Environment. (Springer, Berlin), 358–367. (2014). https://doi.org/10.1007/978-3-662-45283-7_37
Kaur, N., Bahl, M. & Kaur, H. Introduce review on: image fusion using wavelet and curvelet transform (IJCSIT). Int. J. Comp. Sci. Inf. Technol. 5 (2), 2467–2470 (2014).
daCunha, A. L., Zhou, J. & Do, M. N. The nonsubsampled contourlet transform: theory, design and applications. IEEE Trans. Image Process. 15 (10), 3089–3101 (2006).
Rockinger, O. (ed Fechner) Pixel-level image fusion: the case of image sequences. Proc. SPIE 3374 378–388 (1998).
Petrovic, V. S. & Xydeas, C. S. Gradient-based multiresolution image fusion. IEEE Trans. Image Process. 13 (2), 228–237 (2004).
Li, W., Lin, Q., Wang, K. & Cai, K. Improving medical image fusion method using fuzzy entropy and nonsubsampling contourlet transform. Int. J. Imaging Syst. Technol. 31 (1), 204–214 (2021).
Li, X., Zhou, F., Tan, H., Zhang, W. & Zhao, C. Multi-modal medical image fusion based on joint bilateral filter and local gradient energy. Inf. Sci. 569, 302–325 (2021).
Wang, S. & Shen, Y. Multi-modal image fusion based on saliency guided in NSCT domain. IET Image Proc. 14 (13), 3188–3201 (2020).
Zhu, Z., Zheng, M., Qi, G., Di Wang & Xiang, Y. A phase congruency and local laplacian energy based multi-modality medical image fusion method in NSCT domain. IEEE Access. 7, 20811–20824 (2019).
Li, Q., Wang, W., Chen, G. & Zhao, D. Medical image fusion using segment graph filter and sparse representation. Comput. Biol. Med. 131, 104239 (2021).
Guihong, Q., Dali, Z. & Pingfan, Y. Medical image fusion by wavelet transform modulus maxima. Opt. Express. 9, 184–190 (2001).
Yang, L., Guo, B. L. & Ni, W. Multimodality medical image fusion based on multi-scale geometric analysis of contourlet transform. Neurocomputing 72, 203–211 (2008).
Xingbin Liu, W. & Mei Huiqian:’Multi-modality medical image fusion based on image fusion decomposition framework and non-subsampled shearlet transform’. 2018.
James, A. P. & Dasarathy, B. V. :’Medical Image Fussion:a Survey of the state-of-the-art’, Inf (Fusion, 2014).
Qu, X., Yan, J., Xiao, H. & Zhu, Z. Image fusion algorithm based on Spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Automatica Sinica. 34 (12), 1508–1514 (2008).
Li, X. & Zhao, J. A novel multi-modal medical image fusion algorithm. J. Ambient Intell. Humaniz. Comput. 12 (2), 1995–2002 (2021).
Babu, B. S. & Narayana, M. V. Two stage multi-modal medical image fusion with marine predator algorithm-based cascaded optimal DTCWT and NSST with deep learning. Biomed. Signal Process. Control. 85, 104921 (2023).
Zhang, J., Yu, K., Wen, Z., Qi, X. & Paul, A. K. 3D reconstruction for motion blurred images using deep learning-based intelligent systems. CMC-computers Mater. Continua. 66 (2), 2087–2104 (2021).
Fu, J. et al. SMRFnet: saliency multi-scale residual fusion network for grayscale and Pseudo color medical image fusion. Biomed. Signal Process. Control. 100, 107050 (2025).
Liu, Q. et al. Contour-maintaining-based image adaption for an efficient ambulance service in intelligent transportation systems. IEEE Access. 8, 12644–12654 (2020).
Wei, L. et al. Pixel-level structure awareness for enhancing multi-modal medical image fusion. Biomed. Signal Process. Control. 97, 106694 (2024).
Zhu, H., Gowen, A., Feng, H., Yu, K. & Xu, J. L. Deep Spectral-Spatial features of near infrared hyperspectral images for Pixel-Wise classification of food products. Sensors 20 (18), 5322 (2020).
Kaur, H. et al. Multimodal Medical Image Fusion Utilizing Two-scale Image Decomposition via Saliency Detection. Curr. Med. Imaging 20(1), e15734056260083 (2024).
Wang, G., Li, W., Gao, X., Xiao, B. & Du, J. Multimodal medical image fusion based on multichannel coupled neural P systems and max-cloud models in spectral total variation domain. Neurocomputing 480, 61–75 (2022).
Goyal, S., Singh, V., Rani, A. & Yadav, N. Multimodal image fusion and denoising in NSCT domain using CNN and FOTGV. Biomed. Signal Process. Control. 71, 103214 (2022).
Duan, J., Chen, L. & Philip Chen, C. L. Multifocus image fusion with enhanced linear spectral clustering and fast depth map estimation. Neurocomputing 318, 43–54 (2018).
Vijendran, A. S. & Ramasamy, K. Optimal Segmentation and Fusion of multi-modal Brain Images Using Clustering Based Deep Learning Algorithm Vol. 27, p.100691 (Sensors, 2023).
Singh, P. & Bose, S. S. Ambiguous D-means fusion clustering algorithm based on ambiguous set theory: Special application in clustering of CT scan images of COVID-19. Knowledge-Based Systems 231, 107432 (2021).
Arora, P., Mehta, R. & Ahuja, R. An integration of meta-heuristic approach utilizing kernel principal component analysis for multimodal medical image registration. Cluster Comput., 27(5),1–24 (2024).
Zhang, H., Zuo, X., Zhou, H., Lu, T. & Ma, J. A Robust Mutual-Reinforcing Framework for 3D Multi-Modal Medical Image Fusion Based on Visual-Semantic Consistency. Proc. AAAI Conf. Artif. Intell 38(7), 7087–7095 (2024).
Multimodal medical image dataset. Available at: https://github.com/bsun0802/Zero-Learning-Fast-Medical-Image-Fusion/tree/master/images/MRI-PET (Access date: 26-12-2024).
Multimodal medical image dataset. Available at: https://github.com/ashna111/multimodal-image-fusion-to-detect-brain-tumors/tree/master/dataset. (Access date: 26-12-2024).
Multimodal medical image datasets. Available at: https://www.researchgate.net/figure/Multimodal-medical-image-datasets_fig4_351121182 (Access date: 26-12-2024).
Liang, N. Medical image fusion with deep neural networks. Sci. Rep. 14 (1), 7972 (2024).
Luo, F., Wu, D., Pino, L. R. & Ding, W. A novel multimodel medical image fusion framework with edge enhancement and cross-scale transformer. Sci. Rep. 15 (1), 11657 (2025).
Dinh, P. H. MIF-BTF-MRN: medical image fusion based on the bilateral texture filter and transfer learning with the ResNet-101 network. Biomed. Signal Process. Control. 100, 106976 (2025).
Do, O. C., Luong, C. M., Dinh, P. H. & Tran, G. S. An efficient approach to medical image fusion based on optimization and transfer learning with VGG19. Biomed. Signal Process. Control. 87, 105370 (2024).
Dinh, P. H. & Giang, N. L. Medical image fusion based on transfer learning techniques and coupled neural P systems. Neural Comput. Appl. 36 (8), 4325–4347 (2024).
Dinh, P. H., Vu, V. H. & Giang, N. L. A new approach to medical image fusion based on the improved extended difference-of-gaussians combined with the Coati optimization algorithm. Biomed. Signal Process. Control. 93, 106175 (2024).
Dinh, P. H. A novel approach using the local energy function and its variations for medical image fusion. Imaging Sci. J. 71 (7), 660–676 (2023).
Dinh, P. H. Combining spectral total variation with dynamic threshold neural P systems for medical image fusion. Biomed. Signal Process. Control. 80, 104343 (2023).
Dinh, P. H. Enhancing medical image fusion through advanced decomposition and optimization methods. Digital Signal. Processing, 165, 105315. (2025).
Dinh, P. H. A novel approach based on grasshopper optimization algorithm for medical image fusion. Expert Syst. Appl. 171, 114576 (2021).
Acknowledgements
This scientific paper is derived from a research grant funded by Taibah University, Madinah, Kingdom of Saudi Arabia - with grant number (447151101).
Funding
This scientific paper is derived from a research grant funded by Taibah University, Madinah, Kingdom of Saudi Arabia - with grant number (447151101).
Author information
Authors and Affiliations
Contributions
M.J.K. and M.D. : writing original draft; P.Sr., P.Si., N.K.P., M.M.A. and J.A. : writing, review, and editing.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Khan, M.Z., Diwakar, M., Srivastava, P. et al. Multimodality medical image fusion using directional total variation based linear spectral clustering in NSCT domain. Sci Rep (2026). https://doi.org/10.1038/s41598-025-26916-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-025-26916-y
