Low-altitude small-sized object detection using lightweight feature-enhanced convolutional neural network

doi:10.23919/JSEE.2021.000073

Journal of Systems Engineering and Electronics ›› 2021, Vol. 32 ›› Issue (4): 841-853.doi: 10.23919/JSEE.2021.000073

• ELECTRONICS TECHNOLOGY • Previous Articles Next Articles

Low-altitude small-sized object detection using lightweight feature-enhanced convolutional neural network

Tao YE^1,*(), Zongyang ZHAO¹(), Jun ZHANG¹(), Xinghua CHAI²(), Fuqiang ZHOU³()

¹ School of Mechanical and Electrical Information Engineering, China University of Mining and Technology-Beijing, Beijing 100083, China
² The 54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China
³ School of Instrumentation Science and Opto-electronics Engineering, Beihang University, Beijing 100083, China

Received:2021-02-19 Online:2021-08-18 Published:2021-09-30
Contact: Tao YE E-mail:ayetao198715@163.com;303616426@qq.com;973974045@qq.com;cxh88_88@163.com;zfq@buaa.edu.cn
About author:|YE Tao was born in 1987. He received his B.S. degree in measurement and control technology and instrumentation from China University of Mining and Technology, Xuzhou, China, in 2009, M.S. degree in mechanical and electronic engineering from China University of Mining and Technology-Beijing, Beijing, China, in 2012, and Ph.D. degree in measurement technology and instruments from the Key Laboratory of Precision Opto-mechatronics Technology of Ministry of Education, Beihang University, Beijing, in 2016. He was an engineer with Beijing Institute of Remote Sensing and Equipment from 2016 to March 2019. He is currently a senior engineer with the School of Mechanical Electronics and Information Engineering, China University of Mining and Technology-Beijing, Beijing. His current research interests include deep learning and traffic detection. E-mail: ayetao198715@163.com||ZHAO Zongyang was born in 1998. He received his B.S. degree in mechanical engineering from China University of Mining and Technology-Beijing, China, in 2020, where he is currently pursuing his M.S. degree with the Department of Mechanical Engineering, School of Mechanical Electronic and Information Engineering. His current research interests include deep learning and railway object detection. E-mail: 303616426@qq.com||ZHANG Jun was born in 1998. He received his B.S. degree in mechanical engineering from China University of Mining and Technology-Beijing in 2020, Beijing, China. He is currently pursuing his M.S. degree in mechanical engineering at China University of Mining and Technology-Beijing. His research interests include artificial intelligence, deep learning, and multiple-object tracking. E-mail: 973974045@qq.com||CHAI Xinghua was born in 1986. He received his B.S. degree from Hohai University in 2008, M.S. degree from Beijing Information Science and Technology University in 2013, and Ph.D. degree from Beihang University in 2017. He is working as a senior engineer with the 54th Research Institute of the China Electronics Technology Group Corporation. His research interests include machine vision, computer vision, and multi-agent systems. E-mail: cxh88_88@163.com||ZHOU Fuqiang was born in 1972. He received his B.S., M.S., and Ph.D. degrees in instrument, measurement, and test technology from Tianjin University, China, in 1994, 1997, and 2000, respectively. He joined the School of Automation Science and Electrical Engineering, Beihang University, China, as a post-doctoral research fellow, in 2000. He is currently a professor with the School of Instrumentation and Opto-electronics Engineering, Beihang University. His research interests include precision vision measurement, 3D vision sensors, image recognition, and optical metrology. E-mail: zfq@buaa.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (52075027) and the Fundamental Research Funds for the Central Universities (2020XJJD03)

Abstract

Abstract:

Unauthorized operations referred to as “black flights” of unmanned aerial vehicles (UAVs) pose a significant danger to public safety, and existing low-attitude object detection algorithms encounter difficulties in balancing detection precision and speed. Additionally, their accuracy is insufficient, particularly for small objects in complex environments. To solve these problems, we propose a lightweight feature-enhanced convolutional neural network able to perform detection with high precision detection for low-attitude flying objects in real time to provide guidance information to suppress black-flying UAVs. The proposed network consists of three modules. A lightweight and stable feature extraction module is used to reduce the computational load and stably extract more low-level feature, an enhanced feature processing module significantly improves the feature extraction ability of the model, and an accurate detection module integrates low-level and advanced features to improve the multiscale detection accuracy in complex environments, particularly for small objects. The proposed method achieves a detection speed of 147 frames per second (FPS) and a mean average precision (mAP) of 90.97% for a dataset composed of flying objects, indicating its potential for low-altitude object detection. Furthermore, evaluation results based on microsoft common objects in context (MS COCO) indicate that the proposed method is also applicable to object detection in general.

Key words: unmanned aerial vehicle (UAV), deep learning, lightweight network, object detection, low-attitude

Tao YE, Zongyang ZHAO, Jun ZHANG, Xinghua CHAI, Fuqiang ZHOU. Low-altitude small-sized object detection using lightweight feature-enhanced convolutional neural network[J]. Journal of Systems Engineering and Electronics, 2021, 32(4): 841-853.

Figures/Tables 17

Fig 1

Fig 2

Fig 3

Fig 4

Fig 5

Fig 6

Fig 7

Fig 8

Table 1

Fig 9

Fig 10

Table 2

Fig 11

Fig 12

Table 3

Table 4

Fig 13

References 42

1	ZHANG Y, SI G Y, WANG Y Z Simulation of unmanned aerial vehicle swarm electromagnetic operation concept. Systems Engineering and Electronics, 2020, 42 (7): 1510- 1518.
2	MA Q, ZHU B, CHENG Z D, et al Detection and recognition method of fast low-altitude unmanned aerial vehicle based on dual channel. Acta Optica Sinica, 2019, 39 (12): 105- 115.
3	DONG Q, ZOU Q H. Visual UAV detection method with online feature classification. Proc. of the IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference, 2017: 429–432.
4	HU Y Y, WU X J, ZHENG G D, et al Object detection of UAV for anti-UAV based on improved YOLO v3. Chinese Control Conference, 2019, 8386- 8390.
5	ZHANG H, CAO C H, XU L W, et al A UAV detection algorithm based on an artificial neural network. IEEE Access, 2018, 6 (1): 24720- 24728.
6	ZHAO W C, ZHANG Q, LI H, et al. Low-altitude UAV detection method based on one-staged detection framework. Proc. of the 2nd International Conference on Advances in Computer Technology, Information Science and Communications, 2020: 112–117.
7	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770–778.
8	LOW D G Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60 (2): 91- 110. doi: 10.1023/B:VISI.0000029664.99615.94
9	BAY H, TUYTELAARS T, GOOL L V SURF: speeded up robust features. Computer Vision & Image Understanding, 2006, 110 (3): 404- 417.
10	FREUND Y. Experiments with a new boosting algorithm. Proc. of the 13th International Conference on Machine Learning, 1996: 148–156.
11	WU S Q, NAGAHASHI H Parameterized AdaBoost: introducing a parameter to speed up the training of real AdaBoost. IEEE Signal Processing Letters, 2014, 21 (6): 687- 691. doi: 10.1109/LSP.2014.2313570
12	WANG X H, ZHANG C, LI C, et al Unmanned aerial vehicles target detection based on bio-inspired visual attention. Aeronautical Science & Technology, 2015, 26 (11): 76- 82.
13	OMID-ZOHOOR A, YOUNG C, TA D, et al Toward always-on mobile object detection: energy versus performance tradeoffs for embedded hog feature extraction. IEEE Trans. on Circuits and Systems for Video Technology, 2018, 28 (5): 1102- 1115. doi: 10.1109/TCSVT.2017.2653187
14	LIU Y, DU H J, YUE Z H, et al Unmanned aerial vehicle detection method based on random forest. Computer Engineering and Applications, 2019, 55 (7): 162- 167.
15	LI Y Z, XIE P C, TANG Z S, et al SVM-based sea-surface small target detection: a false-alarm-rate-controllable approach. IEEE Geoscience and Remote Sensing Letters, 2019, 16 (8): 1225- 1229. doi: 10.1109/LGRS.2019.2894385
16	BAZI Y, MELGANI F Convolutional SVM networks for object detection in UAV imagery. IEEE Trans. on Geoscience and Remote Sensing, 2018, 56 (6): 3107- 3118. doi: 10.1109/TGRS.2018.2790926
17	ZEILER M D, FERGUS R. Visualizing and understanding convolutional neural networks. Proc. of the European Conference on Computer Vision, 2014: 818–833.
18	GIRSHICK R, DONAHUE J, DARRELL T, et al Rich feature hierarchies for accurate object detection and semantic segmentation. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, 580- 587.
19	GIRSHICK R. Fast R-CNN. Proc. of the IEEE International Conference on Computer Vision, 2015: 1440–1448.
20	REN S Q, HE K M, GIRSHICK R, et al Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
21	WANG J Q, CHEN K, YANG S, et al. Region proposal by guided anchoring. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 2965–2974.
22	ANGUELOV D, ERHAN D, SZEGEDY C, et al. SSD: single shot multibox detector. Proc. of the European Conference on the Computer Vision, 2016: 21–37.
23	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779–788.
24	REDOMN J, FARHADI A. YOLO9000: better, faster, stronger. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6517–6525.
25	REDOMN J, FARHADI A. YOLOv3: an incremental improvement. https://arXiv.arg/abs/1804.02767.
26	BOCHKOVSKIY A, WANG C Y, MARK LIAO H Y. YOLOv4: optimal speed and accuracy of object detection. https://arXiv.arg/abs/2004.10934.
27	ZHOU X, WANG D, KRAHENBUHL P. Objects as points. https://arXiv.arg/abs/1904.07850.
28	TIAN Z, SHEN C H, CHEN H, et al. FCOS: fully convolutional one-stage object detection. Proc. of the IEEE/CVF International Conference on Computer Vision, 2019: 9626–9635.
29	LI Q Z, XIONG R, WANG R P, et al Research on real-time UAV recognition method based on SSD algorithm. Ship Electronic Engineering, 2019, 39 (5): 30- 35.
30	KRIZHEVSKY A, SUTSKEVER I, GEOFFREY E H ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60 (6): 84- 90. doi: 10.1145/3065386
31	RANGA A, TYAGI A, BERG A C, et al. DSSD: deconvolutional single shot detector. https://arXiv.arg/abs/1701.06659.
32	ZENG J X, XIONG J L, FU X, et al ReFPN-FCOS: one-stage object detection for feature learning and accurate localization. IEEE Access, 2020, 8, 225052- 225063. doi: 10.1109/ACCESS.2020.3044564
33	FAN J X, LI D W, WANG H L, et al. UAV low altitude flight threat perception based on improved SSD and KCF. Proc. of the IEEE 15th International Conference on Control and Automation, 2019: 121–132.
34	FAN S Q, ZHU F H, CHEN S C, et al FII-CenterNet: an anchor-free detector with foreground attention for traffic object detection. IEEE Trans. on Vehicular Technology, 2021, 70 (1): 121- 132. doi: 10.1109/TVT.2021.3049805
35	MA Q, ZHU B, ZHANG H W, et al Low-altitude UAV detection and recognition method based on optimized YOLOv3. Laser & Optoelectronics Progress, 2019, 56 (20): 279- 286.
36	CUI Y P, WANG Y H, HU J W Detection method for a dynamic small target using the improved YOLOv3. Journal of Xidian University, 2020, 47 (3): 1- 7.
37	ARTUUR D, VASSILVITSKII S. k-means++: the advantages of careful seeding. Proc. of the 18th Annual ACM-SIAM Symposium on Discrete Algorithms, 2007: 1027–1035.
38	WEI X K, WEI D H, SUO D, et al Multi-object defect identification for railway track line based on image processing and improved YOLOv3 model. IEEE Access, 2020, 8, 61973- 61988. doi: 10.1109/ACCESS.2020.2984264
39	WANG C Y, LIAO H Y M, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020: 1571–1580.
40	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition. computer vision and pattern recognition. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770–778.
41	XIE S N, GIRSHICK R, DOLLAR P, et al. Aggregated residual transformations for deep neural networks. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5987–5995.
42	TSUNG Y L, MICHAEL M, BELONGIE S, et al. Microsoft COCO: common objects in context. Proc. of the European Conference on Computer Vision, 2014: 740–755.

Feature-map size	Receptive field size	Anchor box size
13 × 13	32 × 32	320, 321, 196, 237, 155, 128
26 × 26	16 × 16	98, 186, 82, 86, 55, 126
52 × 52	8 × 8	56, 64, 33, 48, 13, 26

Component	Faster	SSD	Mbv2-SSD	YOLOv3	YOLOv3-tiny	FOCS	Center Net	YOLOv4	YOLOv4-tiny	LSL-Net
Bird	0.897 6	0.835 4	0.711 5	0.941 0	0.803 9	0.937 6	0.934 4	0.962 8	0.836 0	0.905 4
Kite	0.785 1	0.729 2	0.679 4	0.861 7	0.685 6	0.880 7	0.867 1	0.916 7	0.757 4	0.871 4
UAV	0.925 6	0.912 5	0.816 5	0.951 3	0.894 0	0.967 5	0.957 0	0.981 3	0.934 4	0.952 3
mAP	0.869 5	0.825 7	0.735 8	0.918 0	0.794 5	0.928 6	0.919 5	0.953 6	0.842 6	0.909 7
FPS	12	43	81	77	334	42	89	49	287	147

Component	YOLOv4-tiny	EA-Net	LA-Net	LE-Net	LSL-Net
LSM	—	—	√	√	√
EFM	—	√	—	√	√
ADM	—	√	√	—	√
Bird	0.836 0	0.900 5	0.871 7	0.831 2	0.905 4
Kite	0.757 4	0.868 3	0.856 2	0.806 2	0.871 4
UAV	0.934 4	0.949 8	0.940 6	0.931 5	0.952 3
mAP	0.842 6	0.906 2	0.889 5	0.856 3	0.909 7
FPS	287	141	169	253	147

Method	Size	mAP/%	FPS
Faster R-CNN[21]	—	39.8	9
SSD[22]	300×300	25.1	43
SSD[22]	512×512	28.8	22
YOLOv3-SPP[25]	608×608	36.2	20
YOLOv4[26]	608×608	43.5	33
CenterNet[27]	—	41.6	28
FCOS[28]	—	44.7	—
LSL-Net(ours)	416×416	38.4	135
LSL-Net(ours)	512×512	39.1	126
LSL-Net(ours)	608×608	40.3	118

[1]	Honghong ZHANG, Xusheng GAN, Shuangfeng LI, Zhiyuan CHEN. UAV safe route planning based on PSO-BAS algorithm [J]. Journal of Systems Engineering and Electronics, 2022, 33(5): 1151-1160.
[2]	Yangjun GAO, Guangyun LI, Zhiwei LYU, Lundong ZHANG, Zhongpan LI. Improved adaptively robust estimation algorithm for GNSS spoofer considering continuous observation error [J]. Journal of Systems Engineering and Electronics, 2022, 33(5): 1237-1248.
[3]	Yuan ZENG, Wenbin LU, Bo YU, Shifei TAO, Haosu ZHOU, Yu CHEN. Improved IMM algorithm based on support vector regression for UAV tracking [J]. Journal of Systems Engineering and Electronics, 2022, 33(4): 867-876.
[4]	Jun HAN, Weixing LI, Kai FENG, Feng PAN. Vision-based aerial image mosaicking algorithm with object detection [J]. Journal of Systems Engineering and Electronics, 2022, 33(2): 259-268.
[5]	Dongju CAO, Wendong YANG, Hui CHEN, Yang WU, Xuanxuan TANG. Energy efficiency maximization for buffer-aided multi-UAV relaying communications [J]. Journal of Systems Engineering and Electronics, 2022, 33(2): 312-321.
[6]	Haifen YANG, Hao ZHANG, Houjun WANG, Zhengyang GUO. A novel approach for unlabeled samples in radiation source identification [J]. Journal of Systems Engineering and Electronics, 2022, 33(2): 354-359.
[7]	Yangyang JIANG, Yan GAO, Wenqi SONG, Yue LI, Quan QUAN. Bibliometric analysis of UAV swarms [J]. Journal of Systems Engineering and Electronics, 2022, 33(2): 406-425.
[8]	Jinqiang HU, Husheng WU, Renjun ZHAN, Rafik MENASSEL, Xuanwu ZHOU. Self-organized search-attack mission planning for UAV swarm based on wolf pack hunting behavior [J]. Journal of Systems Engineering and Electronics, 2021, 32(6): 1463-1476.
[9]	Ziquan YU, Youmin ZHANG, Bin JIANG. PID-type fault-tolerant prescribed performance control of fixed-wing UAV [J]. Journal of Systems Engineering and Electronics, 2021, 32(5): 1053-1061.
[10]	Juan Antonio VAZQUEZ TREJO, Adrien GUENARD, Manuel ADAM-MEDINA, Jean-Christophe PONSART, Laurent CIARLETTA, Damiano ROTONDO, Didier THEILLIOL. Event-triggered leader-following formation control for multi-agent systems under communication faults: application to a fleet of unmanned aerial vehicles [J]. Journal of Systems Engineering and Electronics, 2021, 32(5): 1014-1022.
[11]	Zhao SUN, Chao MA, Liang WANG, Ran MENG, Shanshan PEI. A deep learning-based binocular perception system [J]. Journal of Systems Engineering and Electronics, 2021, 32(1): 7-20.
[12]	Hongyin SHI, Yue LIU, Jianwen GUO, Mingxin LIU. ISAR autofocus imaging algorithm for maneuvering targets based on deep learning and keystone transform [J]. Journal of Systems Engineering and Electronics, 2020, 31(6): 1178-1185.
[13]	Chuan LIN, Qing CHANG, Xianxu LI. Uplink NOMA signal transmission with convolutional neural networks approach [J]. Journal of Systems Engineering and Electronics, 2020, 31(5): 890-898.
[14]	Gaofeng WU, Kaifang WAN, Xiaoguang GAO, Xiaowei FU. Placement of unmanned aerial vehicles as communication relays in two-tiered multi-agent system: clustering based methods [J]. Journal of Systems Engineering and Electronics, 2020, 31(2): 231-242.
[15]	Zhen XU, Enze ZHANG, Qingwei CHEN. Rotary unmanned aerial vehicles path planning in rough terrain based on multi-objective particle swarm optimization [J]. Journal of Systems Engineering and Electronics, 2020, 31(1): 130-141.

Low-altitude small-sized object detection using lightweight feature-enhanced convolutional neural network

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 17

References 42

Related Articles 15

Recommended Articles

Metrics

Comments