Adaptive Deep Image Fusion through PSO Segmentation and Sparse LSQR Optimization

Document Type : Original Article

Authors

1 MSC Student, Faculty of Electrical and Computer Engineering, Hamedan University of Technology, Hamedan, Iran. Email: Hediehnoo2020@gmail.com

2 Corresponding author, Associate Professor, Faculty of Electrical and Computer Engineering, Hamedan University of Technology, Hamedan, Iran. Email: h.doosti@hut.ac.ir

10.22091/jemsc.2026.15317.1348

Abstract

This study presents a region-adaptive hybrid fusion framework that integrates unsupervised clustering, deep convolutional modeling, and sparse numerical optimization to enhance multispectral satellite image fusion. Unlike classical pansharpening approaches, which are computationally efficient but inherently global and unable to adequately handle spatial–spectral variability, and purely deep-learning-based models that often overgeneralize across heterogeneous scenes, the proposed method introduces a region-aware strategy to preserve fine structures and spectrally weak areas. By employing Particle Swarm Optimization–based unsupervised segmentation, the framework dynamically partitions the image into spectrally homogeneous regions, enabling localized CNN/ResNet-based fusion that prevents spectral dilution near region boundaries. The independently fused regions are subsequently unified through LSQR-based global optimization, which enforces structural coherence and minimizes reconstruction error without reliance on large labeled datasets. Experimental evaluations on IKONOS datasets demonstrate that the proposed framework consistently outperforms classical and state-of-the-art fusion methods in terms of UIQI, RMSE, CC, and ERGAS metrics, confirming its robustness, adaptability, and effectiveness for high-fidelity multispectral image enhancement in complex remote-sensing environments.

Keywords

Main Subjects

References

Aiazzi, B., Baronti, S., Pippi, I., & Alparone, L. (2003). Decision-driven pyramid fusion of multispectral and panchromatic images. In Geoinformation For European-Wide Integration (pp. 273-278). Millpress Science Publisher, PO BOX 84118, 3009 CC Rotterdam. Netherland. https://doi.org/10.1080/01431160512331314056
Burt, P. J., & Adelson, E. H. (1987). The Laplacian pyramid as a compact image code. In Readings in computer vision (pp. 671-679). Morgan Kaufmann. http://dx.doi.org/10.1016/B978-0-08-051581-6.50065-9
Chao, W., Rui, Y., Sun, Y., Jiang, Y., & Lin, X. (2018). A new method of multi-focus image fusion using laplacian operator and region optimization. Journal of Computer and Communications, 6(5), 106-118. http://dx.doi.org/10.4236/jcc.2018.65009
Fang, Y., Cai, Y., & Fan, L. (2023). SDRCNN: A single-scale dense residual connected convolutional neural network for pansharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16, 6325-6338. http://dx.doi.org/10.1109/JSTARS.2023.3292320
Golub, G. H., & Van Loan, C. F. (2013). Matrix computations. JHU press. http://dx.doi.org/10.56021/9781421407944
Golub, G., & Kahan, W. (1965). Calculating the singular values and pseudo-inverse of a matrix. Journal of the Society for Industrial and Applied Mathematics, Series B: Numerical Analysis, 2(2), 205-224. http://dx.doi.org/10.1137/0702016
He, L., Rao, Y., Li, J., Chanussot, J., Plaza, A., Zhu, J., & Li, B. (2019). Pansharpening via detail injection based convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(4), 1188-1204. https://doi.org/10.1109/JSTARS.2019.2898574
Kohonen, T. (2012). Self-organizing maps (Vol. 30). Springer Science & Business Media. http://dx.doi.org/10.1007/978-3-642-97966-8
Lemaire, V., & Clérot, F. (2005). The many faces of a Kohonen map a case study: SOM-based clustering for on-line fraud behavior classification. In S. K. Halgamuge & L. Wang (Eds.), Classification and clustering for knowledge discovery (pp. 1-13). Springer Berlin Heidelberg. http://dx.doi.org/10.1007/11011620_1
Li, X. J., Yan, H. W., Yang, S. W., Kang, L., & Lu, X. M. (2018). Multispectral pansharpening approach using pulse-coupled neural network segmentation. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42, 961-965. http://dx.doi.org/10.5194/isprs-archives-XLII-3-961-2018
Li, X., Xu, F., Lyu, X., Tong, Y., Chen, Z., Li, S., & Liu, D. (2020). A remote-sensing image pan-sharpening method based on multi-scale channel attention residual network. Ieee Access, 8, 27163-27177. http://dx.doi.org/10.1109/ACCESS.2020.2971502
Li, Y., & Zhao, J. (2020). A novel medical image fusion method using multi-channel pulse coupled neural networks. IEEE Access, 8, 157572-157586. http://dx.doi.org/10.1109/ACCESS.2020.3019426
Liu, Q., Han, L., Tan, R., Fan, H., Li, W., Zhu, H. & Liu, S. (2021). Hybrid attention based residual network for pansharpening. Remote Sensing, 13(10), 1962. http://dx.doi.org/10.3390/rs13101962
Liu, X., Liu, Q., & Wang, Y. (2020). Remote sensing image fusion based on two-stream fusion network. Information Fusion, 55, 1-15. http://dx.doi.org/10.1016/j.inffus.2019.07.010
Ma, M., Jiang, Y., Zhao, M., Li, J., & Zhang, W. (2025). HetSSNet: Spatial-Spectral heterogeneous graph learning network for panchromatic and multispectral images fusion. arXiv preprint arXiv:2502.04623. https://doi.org/10.48550/arXiv.2502.04623
Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sensing, 8(7), 594. http://dx.doi.org/10.3390/rs8070594
Paige, C. C., & Saunders, M. A. (1982). LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software (TOMS), 8(1), 43-71. http://dx.doi.org/10.1145/355984.355989
Quan, W., & Guo, W. (2023). DTEMPan: Dual texture-edge maintaining transformer for pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 61, 1-16. http://dx.doi.org/10.1109/TGRS.2023.3335363
Raghavan, K. R. (1994). Inverse and optimum drive problems in ultrasonic imaging of soft tissue elasticity. University of Michigan. https://dx.doi.org/10.1109/NSSMIC.1993.373621
Rao, Y., He, L., & Zhu, J. (2017, May). A residual convolutional neural network for pan-shaprening. In 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP) (pp. 1-4). IEEE. http://dx.doi.org/10.1109/RSIP.2017.7958807
Shen, H., Jiang, M., Li, J., Yuan, Q., Wei, Y., & Zhang, L. (2019). Spatial–spectral fusion by combining deep learning and variational model. IEEE Transactions on Geoscience and Remote Sensing, 57(8), 6169-6181. http://dx.doi.org/10.1109/TGRS.2019.2904659
Shen, H., Meng, X., & Zhang, L. (2016). An integrated framework for the spatio–temporal–spectral fusion of remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 54(12), 7135-7148. http://dx.doi.org/10.1109/TGRS.2016.2596290
Wang, W., Liu, H., & Xie, G. (2021). Pansharpening of WorldView-2 data via graph regularized sparse coding and adaptive coupled dictionary. Sensors, 21(11), 3586. http://dx.doi.org/10.3390/s21113586
Wang, W., Zhou, Z., Liu, H., & Xie, G. (2021). MSDRN: Pansharpening of multispectral images via multi-scale deep residual network. Remote Sensing, 13(6), 1200. http://dx.doi.org/10.3390/rs13061200
Yang, Y., Tu, W., Huang, S., & Lu, H. (2020). PCDRN: Progressive cascade deep residual network for pansharpening. Remote Sensing, 12(4), 676. http://dx.doi.org/10.3390/rs12040676
Yuan, Q., Zhang, L., & Shen, H. (2013). Hyperspectral image denoising with a spatial–spectral view fusion strategy. IEEE Transactions on Geoscience and Remote Sensing, 52(5), 2314-2325. http://dx.doi.org/10.1109/TGRS.2013.2259245
Zhang, T. J., Deng, L. J., Huang, T. Z., Chanussot, J., & Vivone, G. (2022). A triple-double convolutional neural network for panchromatic sharpening. IEEE Transactions on Neural Networks and Learning Systems, 34(11), 9088-9101. http://dx.doi.org/10.1109/TNNLS.2022.3155655
Tavakkoli-Moghaddam, R., Akbari, A. H., Tanhaeean, M., Moghdani, R., Gholian-Jouybari, F., & Hajiaghaei-Keshteli, M. (2024). Multi-objective boxing match algorithm for multi-objective optimization problems. Expert Systems with Applications, 239, 122394. https://doi.org/10.1016/j.eswa.2023.122394 
Send comment about this article
CAPTCHA Image

Files

Share

How to cite

Statistics