Fast image super-resolution algorithm based on multi-resolution dictionary learning and sparse representation

doi:10.21629/JSEE.2018.03.04

Journal of Systems Engineering and Electronics ›› 2018, Vol. 29 ›› Issue (3): 471-482.doi: 10.21629/JSEE.2018.03.04

• Electronics Technology • Previous Articles Next Articles

Fast image super-resolution algorithm based on multi-resolution dictionary learning and sparse representation

Wei ZHAO^1,*(), Xiaofeng BIAN¹(), Fang HUANG¹(), Jun WANG¹(), Mongi A ABIDI²()

¹ School of Electronics and Information Engineering, Beihang University, Beijing 100191, China
² Department of Electrical Engineering and Computer Science, University of Tennessee, Knoxville TN 37996, USA

Received:2017-01-24 Online:2018-06-28 Published:2018-07-02
Contact: Wei ZHAO E-mail:zhaowei203@buaa.edu.cn;xiaofengb@buaa.edu.cn;huangfang@buaa.edu.cn;wj203@buaa.edu.cn;abidi@utk.edu
About author:ZHAO Wei was born in 1972. She received her B.S., M.S. and Ph.D. degrees, all from School of Automatic Control of Northwestern Polytechnical University, Xi'an, China. Then she did postdoctoral research in Beihang University, and now she is an associate professor there. Her main research interests are digital image processing, automatic target recognition, signal processing in wireless sensor network and information fusion. E-mail: zhaowei203@buaa.edu.cn|BIAN Xiaofeng was born in 1993. He received his B.E. degree in Beihang University, China, in 2011. He is currently a master candidate in School of Electronic and Information Engineering of Beihang University, China. His research interest is image processing. E-mail: xiaofengb@buaa.edu.cn|HUANG Fang was born in 1991. She received her B.E. degree in Dalian University of Technology, China, in 2010. She is currently a master candidate in School of Electronic and Information Engineering of Beihang University, China. Her research interest is image processing. E-mail: huangfang@buaa.edu.cn|WANG Jun was born in 1972. He received his B.S. degree from Northwestern Polytechnical University, Xi'an, China, in 1995, and M.S. and Ph.D. degrees from Beihang University, Beijing, China, in 1998 and 2001, respectively. He is currently a professor in School of Electronic and Information Engineering, Beihang University. He is interested in signal processing, DSP/FPGA real-time architecture, target recognition and tracking, and so on. E-mail: wj203@buaa.edu.cn|ABIDI Mongi A. was born in 1955. He holds the life-time Cooke-Eversole professorship in the College of Engineering at the University of Tennessee where he served as a faculty since 1987. He graduated over 75 Masters and Ph.D. students and managed over million of external funding in the areas of image processing and robotics. He has published over 320 papers and is co-author or co-editor of four books. His main research interests are data fusion in robotics and machine intelligence, 3D imaging, face identification, and color image processing and applications. E-mail: abidi@utk.edu

Abstract

Abstract:

Sparse representation has attracted extensive attention and performed well on image super-resolution (SR) in the last decade. However, many current image SR methods face the contradiction of detail recovery and artifact suppression. We propose a multi-resolution dictionary learning (MRDL) model to solve this contradiction, and give a fast single image SR method based on the MRDL model. To obtain the MRDL model, we first extract multi-scale patches by using our proposed adaptive patch partition method (APPM). The APPM divides images into patches of different sizes according to their detail richness. Then, the multiresolution dictionary pairs, which contain structural primitives of various resolutions, can be trained from these multi-scale patches. Owing to the MRDL strategy, our SR algorithm not only recovers details well, with less jag and noise, but also significantly improves the computational efficiency. Experimental results validate that our algorithm performs better than other SR methods in evaluation metrics and visual perception.

Key words: single image super-resolution (SR), sparse representation, multi-resolution dictionary learning (MRDL), adaptive patch partition method (APPM)

Wei ZHAO, Xiaofeng BIAN, Fang HUANG, Jun WANG, Mongi A ABIDI. Fast image super-resolution algorithm based on multi-resolution dictionary learning and sparse representation[J]. Journal of Systems Engineering and Electronics, 2018, 29(3): 471-482.

Figures/Tables 11

Fig 1

Fig 2

Fig 3

Table 1

Fig 4

Table 2

Fig 5

Fig 6

Fig 7

Table 3

Fig 8

References 26

1	ZHU Z, GUO F, YU H, et al. Fast single image superresolution via self-example learning and sparse representation. IEEE Trans. on Multimedia, 2014, 16 (8): 2178- 2190. doi: 10.1109/TMM.2014.2364976
2	FU C H, CHEN H, ZHANG H, et al. Single image super resolution based on sparse representation and adaptive dictionary selection. Proc. of the 19th International Conference on Digital Signal Processing, 2014, 449- 453. doi: 10.1109/ICDSP.2014.6900704
3	VILLENA S, VEGA M, BABACAN S D, et al. Bayesian combination of sparse and non-sparse priors in image super resolution. Digital Signal Processing, 2013, 23 (2): 530- 541.
4	HE L, QI H, ZARETZKI R. Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2013, 345- 352.
5	ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations. Proc. of the International Conference on Curves and Surfaces, 2012, 711- 730.
6	WANG S L, ZHANG L, LIANG Y, et al. Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2012, 2216- 2223.
7	DONG W, ZHANG L, SHI G, et al. Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. on Image Processing, 2011, 20 (7): 1838- 1857. doi: 10.1109/TIP.2011.2108306
8	BABACAN S D, MOLINA R, KATSAGGELOS A K. Variational Bayesian super resolution. IEEE Trans. on Image Processing, 2011, 20 (4): 984- 999. doi: 10.1109/TIP.2010.2080278
9	YANG J, WRIGHT J, HUANG T, et al. Image super-resolution via sparse representation. IEEE Trans. on Image Processing, 2010, 19 (11): 2861- 2873. doi: 10.1109/TIP.2010.2050625
10	YANG J, WRIGHT J, HUANG T, et al. Image super-resolution as sparse representation of raw image patches. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2008, 1- 8.
11	HE Y, YAP K H, CHEN L, et al. A nonlinear least square technique for simultaneous image registration and superresolution. IEEE Trans. on Image Processing, 2007, 16 (11): 2830- 2841. doi: 10.1109/TIP.2007.908074
12	DAI S, HAN M, XU W, et al. Soft edge smoothness prior for alpha channel super resolution. Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2007, 1- 8.
13	LI X, ORCHARD M T. New edge-directed interpolation. IEEE Trans. on Image Processing, 2001, 10 (10): 1521- 1527. doi: 10.1109/83.951537
14	UR H, GROSS D. Improved resolution from subpixel shifted pictures. Cvgip Graphical Models & Image Processing, 1992, 54 (2): 181- 186.
15	KEYS R G. Cubic convolution interpolation for digital image processing. IEEE Trans. on Acoustics Speech & Signal Processing, 1982, 29 (6): 1153- 1160.
16	YANG C Y, YANG M H. Fast direct super-resolution by simple functions. Proc. of the IEEE International Conference on Computer Vision, 2013, 561- 568.
17	YANG J, LIN Z, COHEN S. Fast image super-resolution based on in-place example regression. Proc. of the IEEE Conference on Computer Vision & Pattern Recognition, 2013, 1059- 1066.
18	YU J, GAO X, TAO D, et al. A unified learning framework for single image super-resolution. IEEE Trans. on Neural Networks & Learning Systems, 2014, 25 (4): 780- 792.
19	ZHU Y, ZHANG Y, YUILLE A L. Single image superresolution using deformable patches. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, 2917- 2924.
20	YEGANLI F, NAZZAL M, OZKARAMANLI H. Image super-resolution via sparse representation over multiple learned dictionaries based on edge sharpness and gradient phase angle. Signal Image & Video Processing, 2015, 9 (1): 285- 293.
21	TRINH D H, LUONG M, DIBOS F, et al. Novel examplebased method for super-resolution and denoising of medical images. IEEE Trans. on Image Processing, 2014, 23 (4): 1882- 1895.
22	LIN D, TANG X. Coupled space learning of image style transformation. Proc. of the IEEE International Conference on Computer Vision, 2005, 2, 1699- 1706. doi: 10.1109/ICCV.2005.65
23	KANUNGO T, MOUNT D M, NETANYAHU N S, et al. An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans. on Pattern Analysis & Machine Intelligence, 2002, 24 (7): 881- 892.
24	LEE H, BATTLE A, RAINA R, et al. Efficient sparse coding algorithms. Proc. of the Neural Information Processing Systems, 2007, 721- 728.
25	MOORE B. Principal component analysis in linear systems: controllability, observability, and model reduction. IEEE Trans. on Automatic Control, 1981, 26 (1): 17- 32.
26	ZHOU W, CONRAD B A, RAHIM S H, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans. on Image Processing, 2004, 13 (4): 600- 612.

Image	Parameter	Patch size
Image	Parameter	2×2	4×4	8×8	8×84×4	8×84×42×2
Barbara	PSNR/dB	27.63	27.79	27.42	27.84	27.92
128×128	Time/s	0.75	0.38	0.19	0.35	0.30
Cameraman	PSNR/dB	24.23	24.20	23.97	24.43	24.76
128×128	Time/s	0.63	0.34	0.17	0.29	0.25
Lena	PSNR/dB	32.52	32.71	32.52	32.95	33.03
512×512	Time/s	6.08	2.36	1.28	2.27	2.12
Zebra	PSNR/dB	27.69	27.71	26.77	28.55	28.58
391×586	Time/s	5.97	2.09	1.27	2.01	1.88
Monarch	PSNR/dB	30.60	30.78	30.46	31.16	31.80
512×768	Time/s	9.91	3.50	1.84	3.10	2.93

Image	Parameter	Dictionary size
Image	Parameter	128, 128, 128	256, 256, 256	512, 512, 512	128, 256, 512
Cameraman	PSNR/dB	31.76	31.91	32.02	32.00
512×512	Time/s	1.72	1.94	2.09	1.78
Man	PSNR/dB	28.06	28.08	28.09	28.09
512×512	Time/s	1.84	1.98	2.13	1.91
Monarch	PSNR/dB	31.74	31.80	31.89	31.87
512×768	Time/s	2.80	2.93	3.06	2.87
Ppt3	PSNR/dB	25.13	25.23	25.48	25.48
656×529	Time/s	2.53	2.64	2.87	2.55
Zebra	PSNR/dB	28.55	28.57	28.60	28.58
391×586	Time/s	1.73	1.88	1.98	1.80

Image	Parameter	Bicubic	ScSR	Fu et al.	He et al.	Yeganli	Our
Barbara[576×720]	PSNR/dB	26.25	26.72	26.64	26.84	26.89	26.79
	SSIM	0.753	0.772	0.777	0.786	0.789	0.785
	Time/s	0.09	602.91	68.59	1157.83	60.66	3.48
Bridge[512×512]	PSNR/dB	24.40	24.97	24.81	25.01	25.07	25.06
	SSIM	0.649	0.702	0.696	0.705	0.708	0.708
	Time/s	0.03	404.84	44.77	749.18	38.56	2.08
Cameraman[256×256]	PSNR/dB	23.77	24.44	24.32	24.79	24.63	24.91
	SSIM	0.779	0.802	0.799	0.816	0.811	0.821
	Time/s	0.02	93.09	10.67	174.53	9.28	0.75
Flowers[362×500]	PSNR/dB	27.23	28.28	27.91	28.66	28.47	28.64
	SSIM	0.801	0.840	0.827	0.844	0.840	0.841
	Time/s	0.06	274.56	28.58	508.23	23.30	1.31
Foreman[288×352]	PSNR/dB	31.18	32.81	32.08	33.10	33.26	33.19
	SSIM	0.907	0.924	0.915	0.928	0.931	0.930
	Time/s	0.03	140.43	16.27	269.69	12.45	1.02
Lena[256×256]	PSNR/dB	29.94	30.85	30.75	31.36	31.30	31.36
	SSIM	0.843	0.861	0.856	0.869	0.867	0.869
	Time/s	0.03	90.17	9.75	172.76	7.97	0.69
Man[512×512]	PSNR/dB	27.01	27.72	27.58	28.05	27.96	28.09
	SSIM	0.749	0.784	0.778	0.791	0.790	0.792
	Time/s	0.05	394.50	43.34	740.72	33.25	1.91
Monarch[512×768]	PSNR/dB	29.43	30.76	30.27	31.49	31.00	31.87
	SSIM	0.919	0.935	0.925	0.939	0.936	0.943
	Time/s	0.08	586.23	64.45	1~117.58	50.14	2.87
Pepper[512×512]	PSNR/dB	32.39	33.29	33.03	34.00	34.02	34.11
	SSIM	0.870	0.878	0.874	0.885	0.885	0.887
	Time/s	0.05	391.92	42.45	742.14	33.89	1.92
Ppt3[656×529]	PSNR/dB	27.31	24.65	24.66	25.37	25.12	25.48
	SSIM	0.874	0.891	0.886	0.913	0.902	0.914
	Time/s	0.09	453.03	49.98	798.99	43.19	2.55
Zebra[391×586]	PSNR/dB	26.63	28.02	27.43	28.83	28.45	28.58
	SSIM	0.794	0.837	0.829	0.850	0.846	0.849
	Time/s	0.05	350.36	36.59	649.20	29.20	1.80
Comic[361×250]	PSNR/dB	23.12	23.93	23.75	24.19	24.04	24.06
	SSIM	0.699	0.757	0.747	0.768	0.762	0.762
	Time/s	0.03	115.22	13.56	240.58	15.27	0.72
Average	PSNR/dB	27.09	28.04	27.77	28.47	28.35	28.51
	SSIM	0.803	0.832	0.826	0.841	0.838	0.842
	Time/s	0.05	324.77	35.75	610.12	29.76	1.76

[1]	Guisheng WANG, Yequn WANG, Shufu DONG, Guoce HUANG. Multiple transformation analysis for interference separation in TDCS [J]. Journal of Systems Engineering and Electronics, 2022, 33(5): 1064-1078.
[2]	Yuanrong TIAN, Xing WANG. Learning a discriminative high-fidelity dictionary for single channel source separation [J]. Journal of Systems Engineering and Electronics, 2021, 32(5): 1097-1110.
[3]	Jian WANG, Chunxia QIN, Xiufei ZHANG, Ke YANG, Ping REN. A multi-source image fusion algorithm based on gradient regularized convolution sparse representation [J]. Journal of Systems Engineering and Electronics, 2020, 31(3): 447-459.
[4]	Zhigang XU, Qiang MA, Feixiang YUAN. Single color image super-resolution using sparse representation and color constraint [J]. Journal of Systems Engineering and Electronics, 2020, 31(2): 266-271.
[5]	Jian ZHAO, Chao ZHANG, Shunli ZHANG, Tingting LU, Weiwen SU, Jian JIA. Pre-detection and dual-dictionary sparse representation based face recognition algorithm in non-sufficient training samples [J]. Journal of Systems Engineering and Electronics, 2018, 29(1): 196-202.
[6]	Jingjing Cai, Dan Bao, and Peng Li. DOA estimation via sparse recovering from the smoothed covariance vector [J]. Systems Engineering and Electronics, 2016, 27(3): 555-561.
[7]	Xiang Wang, Zhitao Huang, and Yiyu Zhou. Underdetermined DOA estimation and blind separation of non-disjoint sources in time-frequency domain based on sparse representation method [J]. Journal of Systems Engineering and Electronics, 2014, 25(1): 17-25.
[8]	Jiajia Zhao, Zhengyuan Tang, Jie Yang, and Erqi Liu. Infrared small target detection using sparse representation [J]. Journal of Systems Engineering and Electronics, 2011, 22(6): 897-904.
[9]	Yi Qingming. Blind source separation by weighted K-means clustering [J]. Journal of Systems Engineering and Electronics, 2008, 19(5): 882-887.

Fast image super-resolution algorithm based on multi-resolution dictionary learning and sparse representation

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 11

References 26

Related Articles 9

Recommended Articles

Metrics

Comments