Monocular depth ordering with occlusion edges extraction and local depth inference

doi:10.21629/JSEE.2019.06.04

Journal of Systems Engineering and Electronics ›› 2019, Vol. 30 ›› Issue (6): 1081-1089.doi: 10.21629/JSEE.2019.06.04

• Electronics Technology • Previous Articles Next Articles

Monocular depth ordering with occlusion edges extraction and local depth inference

Guiling SONG(), Aiwei YU(), Xuejing KANG(), Anlong MING*()

School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2018-12-08 Online:2019-12-20 Published:2019-12-25
Contact: Anlong MING E-mail:expsong@qq.com;2603803139@qq.com;kangxuejing@bupt.edu.cn;mal@bupt.edu.cn
About author:SONG Guiling was born in 1978. He is studying for his Ph.D. degree at the School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China. His research interests are image processing and human computer interaction. E-mail: expsong@qq.com|YU Aiwei was born in 1993. He is studying for his master's degree at the School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China. His research interests are image processing and computer vision. E-mail: 2603803139@qq.com|KANG Xuejing was born in 1984. She received her B.S. and M.S. degrees from Tianjin University of Technology in 2008 and 2012, respectively, and Ph.D. degree in Beijing Institute of Technology. She is currently a lecturer with the School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China. Her current research interests include fractional Fourier transform, image processing and computer vision. E-mail: kangxuejing@bupt.edu.cn|MING Anlong was born in 1979. He received his Ph.D. degree in computer science from the School of Computer Science at Beijing University of Posts and Telecommunications, China, in 2008. His research interests include multimedia systems, computer vision and image processing. E-mail: mal@bupt.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China(61701036);This work was supported by the National Natural Science Foundation of China (61701036)

Abstract

Abstract:

In this paper, a method to infer global depth ordering for monocular images is presented. Firstly a distance metric is defined with color, compactness, entropy and edge features to estimate the difference between pixels and seeds, which can ensure the superpixels to obtain more accurate object contours. To correctly infer local depth relationship, a weighting descriptor is designed that combines edge, T-junction and saliency features to avoid wrong local inference caused by a single feature. Based on the weighting descriptor, a global inference strategy is presented, which not only can promote the performance of global depth ordering, but also can infer the depth relationships correctly between two non-adjacent regions. The simulation results on the BSDS500 dataset, Cornell dataset and NYU 2 dataset demonstrate the effectiveness of the approach.

Key words: superpixel segmentation, depth ordering inference, weighting descriptor

Guiling SONG, Aiwei YU, Xuejing KANG, Anlong MING. Monocular depth ordering with occlusion edges extraction and local depth inference[J]. Journal of Systems Engineering and Electronics, 2019, 30(6): 1081-1089.

Figures/Tables 6

Fig 1

Fig 2

Fig 3

Fig 4

Table 1

Fig 5

References 37

1	MARK N, MUMFORD D. The 2.1-d sketch. Proc. of the IEEE International Conference on Computer Vision, 1990: 138-144.
2	YU C C, LIU Y J, WU M T, et al. A global energy optimization framework for 2.1D sketch extraction from monocular images. Graphical Models, 2014, 76 (5): 507- 521. doi: 10.1016/j.gmod.2014.03.015
3	HONG Z, MEI X, TAO D. Dual-force metric learning for robust distracter-resistant tracker. Proc. of the European Conference on Computer Vision, 2012: 513-527.
4	ZHOU Y, BAI X, LIU W Y, et al. Fusion with diffusion for robust visual tracking. Advances in Neural Information Processing Systems, 2012, 4, 2978- 2986.
5	KARSCH K, LIU C, KANG S B. Depth extraction from video using non-parametric sampling. Proc. of the European Conference on Computer Vision, 2012: 775-788.
6	CHEN X, LI Q, ZHAO D, et al. Occlusion cues for image scene layering. Computer Vision and Image Understanding, 2013, 117 (1): 42- 55. doi: 10.1016/j.cviu.2012.10.001
7	GOULD S, FULTON R, KOLLER D. Decomposing a scene into geometric and semantically consistent regions. Proc. of the International Conference on Computer Vision, 2010: 1-8.
8	LI F F, PERONA P. A bayesian hierarchical model for learning natural scene categories. Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005: 524-531.
9	MA J, ZHOU H, ZHAO J, et al. Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans. on Geoscience and Remote Sensing, 2015, 53 (12): 6469- 6481. doi: 10.1109/TGRS.2015.2441954
10	MA J, QIU W, ZHAO J, et al. Robust L2E estimation of transformation for non-rigid registration. IEEE Trans. on Signal Processing, 2015, 63 (5): 1115- 1129. doi: 10.1109/TSP.2014.2388434
11	MING A, WU T, MA J, et al. Monocular depth-ordering reasoning with occlusion edge detection and couple layers inference. IEEE Intelligent Systems, 2016, 31 (2): 54- 65. doi: 10.1109/MIS.2015.94
12	ACHANTA R, SUSSTRUNK S. Superpixels and polygons using simple non-iterative clustering. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4895-4904.
13	HOIEM D, EFROS A A, HEBERT M. Recovering occlusion boundaries from an image. International Journal of Computer Vision, 2011, 91 (3): 328- 346.
14	FOWLKES C C, MARTIN D R, MALIK J. Local figure-ground cues are valid for natural images. Journal of Vision, 2007, 7 (8): 1- 9. doi: 10.1167/7.8.1
15	ARBELAEZ P, MAIRE M, FOWLKES C, et al. Contour detection and hierarchical image segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2011, 33 (5): 898- 916. doi: 10.1109/TPAMI.2010.161
16	AMER M R, YOUSEFI S, RAICH R, et al. Monocular extraction of 2.1d sketch using constrained convex optimization. International Journal of Computer Vision, 2015, 112 (1): 23- 42.
17	JIA Z. A learning-based framework for depth ordering. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2012: 294-301.
18	ZHOU Y, MA J, MING A, et al. Learning training samples for occlusion edge detection and its application in depth ordering inference. Proc. of the 24th International Conference on Pattern Recognition, 2018, DOI: 10.1109/ICPR.2018.8545622.
19	YU Z, FENG C, LIU M Y, et al. Casenet: deep category-aware semantic edge detection. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1-10.
20	PENG W, YUILLE A. Doc: deep occlusion estimation from a single image. Proc. of the European Conference on Computer Vision, 2016: 546-561.
21	WANG G X, LIANG X H, LI F W B. Doobnet: deep object occlusion boundary detection from an image. Computer Vision and Pattern Recognition, 2018, arXiv:1806.03772.
22	HE L, WANG G, HU Z. Learning depth from single images with deep neural network embedding focal length. IEEE Trans. on Image Processing, 2018, 27 (9): 4676- 4689. doi: 10.1109/TIP.2018.2832296
23	FU H, GONG M, WANG C, et al. Deep ordinal regression network for monocular depth estimation. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 2002-2011.
24	XU D, WANG W, TANG H, et al. Structured attention guided convolutional neural fields for monocular depth estimation. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3917-3925.
25	XU D, OUYANG W, WANG X, et al. Pad-net: multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, DOI: 10.1109/CVPR.2018.00077.
26	XU D, RICCI E, OUYANG W, et al. Multi-scale continuous CRFs as sequential deep networks for monocular depth estimation. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5354-5362.
27	LIU M Y, TUZEL O, RAMALINGAM S, et al. Entropy rate superpixel segmentation. Proc. of the Computer Vision and Pattern Recognition, 2011: 2097-2104.
28	ZHANG Y, LI X, GAO X, et al. A simple algorithm of superpixel segmentation with boundary constrain. IEEE Trans. on Circuits & Systems for Video Technology, 2017, 27 (7): 1502- 1514.
29	SHEN J, DU Y, WANG W, et al. Lazy random walks for superpixel segmentation. IEEE Trans. on Image Processing, 2014, 23 (4): 1451- 1462.
30	ACHANTA R, SUSSTRUNK S. Superpixels and polygons using simple non-iterative clustering. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4895-4904.
31	ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2012, 34 (11): 2274- 2282. doi: 10.1109/TPAMI.2012.120
32	CHENG M M, WARRELL J, LIN W Y, et al. Efficient salient region detection with soft image abstraction. Proc. of the IEEE International Conference on Computer Vision, 2014: 1529-1536.
33	JIA Z, GALLAGHER A, CHANG Y J, et al. A learning-based framework for depth ordering. Proc. of the Computer Vision and Pattern Recognition, 2012, DOI: 10.1109/CVPR.2012.6247688.
34	SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor segmentation and support inference from RGBD images. Proc. of the European Conference on Computer Vision, 2012: 1-14.
35	STUTZ D, HERMANS A, LEIBE B. Superpixels: an evaluation of the state-of-the-art. Computer Vision and Image Understanding, 2017: S1077314217300589.
36	LI Z, CHEN J. Superpixel segmentation using linear spectral clustering. Proc. of the Computer Vision and Pattern Recognition, 2015: 1356-1363.
37	LIU M Y, TUZEL O, RAMALINGAM S, et al. Entropy rate superpixel segmentation. Proc. of the Computer Vision and Pattern Recognition, 2011: 2097-2104.

Dataset	[2]	(ⅰ)	(ⅱ)	(ⅲ)	(ⅳ)	(ⅴ)	Ours
Cornell	79.1	84.6	84.3	91.8	91.4	91.0	92.1
NYU 2	60.3	70.6	69.5	78.9	78.5	78.0	79.6

Monocular depth ordering with occlusion edges extraction and local depth inference

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 6

References 37

Related Articles 0

Recommended Articles

Metrics

Comments