Multi-source image fusion algorithm based on fast weighted guided filter

doi:10.21629/JSEE.2019.05.02

Journal of Systems Engineering and Electronics ›› 2019, Vol. 30 ›› Issue (5): 831-840.doi: 10.21629/JSEE.2019.05.02

• Electronics Technology • Previous Articles Next Articles

Multi-source image fusion algorithm based on fast weighted guided filter

Jian WANG^1,^2,*(), Ke YANG¹(), Ping REN¹(), Chunxia QIN¹(), Xiufei ZHANG¹()

¹ School of Electronics and Information Engineering, Northwestern Polytechnical University, Xi'an 710072, China
² No. 365 Institute, Northwestern Polytechnical University, Xi'an 710065, China

Received:2018-11-12 Online:2019-10-08 Published:2019-10-09
Contact: Jian WANG E-mail:jianwang@nwpu.edu.cn;yk@mail.nwpu.edu.cn;1403147639@mail.nwpu.edu.cn;qin@163.com;921391314@qq.com
About author:WANG Jian was born in 1972. He received his Ph.D. degree in signal and information processing from the Northwestern Polytechnical University in 2005. Now he is an assistant professor at the School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China. His current research interests include UAV intelligent processing technology, UAV ground observation video signal processing technology, and multi-source information intelligent processing technology. E-mail: jianwang@nwpu.edu.cn|YANG Ke was born in 1995. She received her B.S. degree in electronic and information engineering from Taiyuan University of Technology in 2017. She is currently pursing her M.S. degree in the School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China. Her research interests include signal processing and image fusion. E-mail: xgdms yk@mail.nwpu.edu.cn|REN Ping was born in 1993. She received her B.S. degree in electronic and information engineering from Northwestern University in 2016. She is now pursing her M.S. degree in the School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China. Her research interests include signal processing and target recognition. E-mail: 1403147639@mail.nwpu.edu.cn|QIN Chunxia was born in 1995. She received her B.S. degree in electronic and information engineering from Northwestern Normal University in 2017. She is now pursing her M.S. degree in the School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China. Her research interest is signal processing. E-mail: chunxia qin@163.com|ZHANG Xiufei was born in 1989. He received his M.S. degree from the School of Electronics and Information Engineering, Northwestern Polytechnical University in 2017. He is now pursing his Ph.D. degree in the School of Automation, Northwestern Polytechnical University, Xi'an, China. His research interests include signal processing and image fusion. E-mail: 921391314@qq.com
Supported by:
the National Natural Science Foundation of China(61472324);the National Natural Science Foundation of China(61671383);Shaanxi Key Industry Innovation Chain Project(2018ZDCXL-G-12-2);Shaanxi Key Industry Innovation Chain Project(2019ZDLGY14-02-02);This work was supported by the National Natural Science Foundation of China (61472324; 61671383), and Shaanxi Key Industry Innovation Chain Project (2018ZDCXL-G-12-2; 2019ZDLGY14-02-02)

Abstract

Abstract:

In last few years, guided image fusion algorithms become more and more popular. However, the current algorithms cannot solve the halo artifacts. We propose an image fusion algorithm based on fast weighted guided filter. Firstly, the source images are separated into a series of high and low frequency components. Secondly, three visual features of the source image are extracted to construct a decision graph model. Thirdly, a fast weighted guided filter is raised to optimize the result obtained in the previous step and reduce the time complexity by considering the correlation among neighboring pixels. Finally, the image obtained in the previous step is combined with the weight map to realize the image fusion. The proposed algorithm is applied to multi-focus, visible-infrared and multi-modal image respectively and the final results show that the algorithm effectively solves the halo artifacts of the merged images with higher efficiency, and is better than the traditional method considering subjective visual consequent and objective evaluation.

Key words: fast guided filter, image fusion, visual feature, decision map

Jian WANG, Ke YANG, Ping REN, Chunxia QIN, Xiufei ZHANG. Multi-source image fusion algorithm based on fast weighted guided filter[J]. Journal of Systems Engineering and Electronics, 2019, 30(5): 831-840.

Figures/Tables 7

Fig 1

Fig 2

Table 1

Fig 3

Fig 4

Table 2

Quantitative assessment of different image fusion methods"

Source	Norm	MWGF	MST-SR	CBF	LP	HMSD	GF	SIFT	Proposed
Multi-focus image (a)	$MI$ $Q^{{AB}/F} SD$SSIM	1.209 60.757 656.4070.899 3	1.313 30.781 156.3070.904 9	1.234 60.775 555.0830.910 4	1.399 30.556 866.0440.832 1	1.346 80.676 859, 3260.907 6	1.172 10.757 556.3570.901 6	1.438 80.784 756.3320.908 6	$\textbf{4.138 8}$$\textbf{0.796 8}$$\textbf{67.633}$$\textbf{0.919 2}$
Multi-focusimage (b)	$MI$$Q^{{AB} /F} SD$SSIM	6.456 30.698 746.7830.845 7	6.881 10.710 646.7810.791 2	6.962 30.715 345.4360.817 0	6.549 00.710 246.7770.718 1	7.124 50.707 945.9730.797 5	7.832 70.727 146.2150.791 3	6.673 40.712 344.5430.874 3	$\textbf{7.866 6}$$\textbf{0.727 3}$$\textbf{47.603}$$\textbf{0.902 1}$
Multi-modelmedicalimage (c)	$MI$$Q^{{AB}/F} SD$SSIM	0.683 10.565 382.7910.745 8	0.777 10.561 585.0510.676 3	0.717 20.537 172.3440.676 1	0.820 00.391 682.9960.503 2	0.734 70.472 677.7360.676 8	0.761 60.591 977.3270.719 6	0.820 00.608 781.3590.706 7	$\textbf{0.874 7}$$\textbf{0.664 9}$$\textbf{89.813}$$\textbf{0.755 4}$
Multi-modelmedicalimage (d)	$MI$$Q^{{AB}/F} SD$SSIM	4.334 30.493 470.4830.824 3	4.117 50.594 872.7640.805 1	4.027 40.519 262.8930.771 9	3.661 90.591 666.6560.651 3	3.887 50.542 369.8430.764 3	4.102 60.605 767.7840.808 0	4.434 30.574 274.8430.794 3	$\textbf{4.617 6}$$\textbf{0.616 3}$$\textbf{77.171}$$\textbf{0.854 5}$
Visible-infraredimage (e)	$MI$$Q^{{AB} /F} SD$SSIM	1.599 50.482 837.959$\textbf{0.780 0}$	1.793 20.530 033.3180.668 2	1.641 20.406 226.8850.559 1	1.655 00.532 929.2060.633 2	1.565 70.512 633.3640.676 8	1.669 30.524 326.5180.634 6	1.590 20.485 9$\textbf{38.405}$0.698 5	$\textbf{1.810 0}$$\textbf{0.536 0}$30.8320.648 3
Visible-infraredimage (f)	$MI$$Q^{{AB} /F} SD$SSIM	2.553 80.643 80.857 743.109	2.614 90.632 60.757 0$\textbf{50.762}$	$\textbf{2.747 9}$0.594 30.740 542.513	2.703 40.657 60.864 147.400	2.373 20.593 20.821 041.667	2.711 00.664 80.931 140.509	2.280 50.541 10.775 241.590	2.610 7$\textbf{0.665 9}$$\textbf{0.937 1}$44.804

Table 2

Table 3

References 27

1	MA J Y, MA Y, LI C. Infrared and visible image fusion methods and applications:a survey. Information Fusion, 2019, 45, 153- 178. doi: 10.1016/j.inffus.2018.02.004
2	MARVASTI F S, MOSAVI M R, NASIRI M. Flying small target detection in IR images based on adaptive toggle operator. Computer Vision, 2018, 12 (4): 527- 534.
3	ZHANG X N. Noise-robust target recognition of SAR images based on attribute scattering center matching. Remote Sensing Letters, 2019, 10 (2): 186- 194.
4	GHASSEMIAN H. A review of remote sensing image fusion methods. Information Fusion, 2016, 32, 75- 89. doi: 10.1016/j.inffus.2016.03.003
5	ZHU J, JIN W Q, LI L, et al. Multiscale infrared and visible image fusion using gradient domain guided image filtering. Infrared Physics&Technology, 2017, 89, 8- 19.
6	KONG W W, LEI Y, ZHAO H X. Adaptive fusion method of visible light and infrared images based on non-subsampled shearlet transform and fast non-negative matrix factorization. Infrared Physics&Technology, 2014, 67, 161- 172.
7	SHAHDOOSTI H R, MEHRABI A. Multimodal image fusion using sparse representation classification in Tetrolet domain. Digital Signal Processing, 2018, 79, 9- 22. doi: 10.1016/j.dsp.2018.04.002
8	LIU Y, CHEN X, WARD R K, et al. Medical image fusion via convolutional sparsity based morphological component analysis. IEEE Signal Processing Letters, 2019, 26 (3): 485- 489.
9	LIU Y, LIU S P, WANG Z F. Multi-focus image fusion with dense sift. Information Fusion, 2015, 23, 139- 155.
10	LI S T, KANG X D, HU J W, et al. Image matting for fusion of multi-focus images in dynamic scenes. Information Fusion, 2013, 14 (2): 147- 162.
11	BAMAL R, KASANA S S. Dual hybrid medical watermarking using walsh-slantlet transform. Multimedia Tools and Applications, 2019, 78 (13): 17899- 17927. doi: 10.1007/s11042-018-6820-9
12	HUANG Y M, LAN Z B, XU W H. Analysis of sagittal alignment parameters following anterior cervical hybrid decompression and fusion of multilevel cervical spondylotic myelopathy. BMC Musculoskeletal Disorders, 2019, 20 (1): 1- 8. doi: 10.1186/s12891-018-2378-y
13	TOET A. Image fusion by a ratio of low pass pyramid. Pattern Recognization Letters, 1989, 9 (4): 245- 253.
14	LEWIS J J, O'CALLAGHAN R J, NIKOLOV S G, et al. Pixeland region-based image fusion with complex wavelets. Information Fusion, 2007, 8 (2): 119- 130.
15	YANG L, GUO B L, NI W. Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing, 2008, 72 (1/3): 203- 211.
16	LI S T, KANG X D, HU J W, et al. Image fusion with guided filtering". IEEE Trans. on Image Processing: a Publication of the IEEE Signal Processing Society, 2013, 22(7): 2864-2875.
17	HE K M, SUN J, TANG X O. Guided image filtering. IEEE Trans. on Pattern Analysis&Machine Intelligence, 2013, 35(6): 1397-1409.
18	Seella K, UMAMAHESWARA R. Fast weighted guided image filter. International Journal of Advanced Research in Computer and Communication Engineering, 2015, 4 (10): 263- 265.
19	KOU F, CHEN W H, WEN C Y, et al. Gradient domain guided image filtering. IEEE Trans. on Image Processing, 2015, 24(11): 4528-4539.
20	HASSEN R, WANG Z, SALAMA M M. Objective quality assessment for multi-exposure multi-focus image fusion. IEEE Trans. on Image Processing, 2015, 24 (9): 2712- 2724. doi: 10.1109/TIP.2015.2428051
21	LIU Z, BLASCH E, XUE Z Y, et al. Objective assessment of multi-resolution image fusion algorithms for context enhancement in night vision:a comparative study. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2012, 34 (1): 94- 109. doi: 10.1109/TPAMI.2011.109
22	ZHOU Z Q, LI S, WANG B. Multi-scale weighted gradientbased fusion for multi-focus images. Information Fusion, 2014, 20, 60- 72. doi: 10.1016/j.inffus.2013.11.005
23	KUMAR B K S. Image fusion based on pixel significance using cross bilateral filter. Signal, Image and Video Processing, 2015, 9 (5): 1193- 1204. doi: 10.1007/s11760-013-0556-9
24	ZHOU Z Q, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters. Information Fusion, 2016, 30, 15- 26. doi: 10.1016/j.inffus.2015.11.003
25	LIU Y, LIU S P, WANG Z F. Multi-focus image fusion with dense SIFT. Information Fusion, 2015, 23, 139- 155. doi: 10.1016/j.inffus.2014.05.004
26	PETROVIC V, XYDEAS C. On the effects of sensor noise in pixel-level image fusion performance. Proc. of the IEEE 3rd International Conference on Image Fusion, 2000: 14-19.
27	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans. on Image Processing, 2004, 13(4): 600-612.

Expression	Meaning
$MI{=}\sum\limits_{i=0}^{L-1} \sum\limits_{j=0}^{L-1}\sum\limits_{k=0}^{L-1} P(i, j, k)\log_2 \frac{P(i, j, k)}{P(i, j)P(k)}$	It represents the information extracted from the source image.
$Q^{({AB} /F)}(i, j)=$ $\dfrac{\sum\limits_{i=1}^I {\sum\limits_{j=1}^J {Q^{AF}\omega ^A(i, j)+Q^{BF}\omega ^B(i, j)} } }{\sum\limits_{i=1}^I {\sum\limits_{j=1}^J {(\omega ^A(i, j)+\omega ^B(i, j))} } }$	It reflects how much the fused image obtains edge information from the source image.
$SD=\sqrt {\frac{1}{MN}\sum\limits_{m=0}^{M-1} {\sum\limits_{n=0}^{N-1} {[F(m, n)-\mu]^2} } } $	It represents the discreteness of the obtained image compared to the gray mean value.
$SSIM_{AB}^F =\dfrac{SSIM_{AF} +SSIM_{BF} }{2}$	It can approximate the degree of distortion of the image.

[1]	Jian WANG, Chunxia QIN, Xiufei ZHANG, Ke YANG, Ping REN. A multi-source image fusion algorithm based on gradient regularized convolution sparse representation [J]. Journal of Systems Engineering and Electronics, 2020, 31(3): 447-459.
[2]	Dongsheng YANG, Shaohai HU, Shuaiqi LIU, Xiaole MA, Yuchao SUN. Multi-focus image fusion based on block matching in 3D transform domain [J]. Journal of Systems Engineering and Electronics, 2018, 29(2): 415-428.
[3]	Kun Liu1, Lei Guo, and Jingsong Chen. Contourlet transform for image fusion using cycle spinning [J]. Journal of Systems Engineering and Electronics, 2011, 22(2): 353-357.

Multi-source image fusion algorithm based on fast weighted guided filter

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 7

References 27

Related Articles 3

Recommended Articles

Metrics

Comments

Method	MWGF	MST-SR	CBF	LP	HMSD	GF	SIFT	Proposed
Time	3.7	8.43	8.9	2.3	1.14	0.2	6.25	2.01