Using approximate dynamic programming for multi-ESM scheduling to track ground moving targets

doi:10.21629/JSEE.2018.01.08

Journal of Systems Engineering and Electronics ›› 2018, Vol. 29 ›› Issue (1): 74-85.doi: 10.21629/JSEE.2018.01.08

• Systems Engineering • Previous Articles Next Articles

Using approximate dynamic programming for multi-ESM scheduling to track ground moving targets

Kaifang WAN(), Xiaoguang GAO*(), Bo LI(), Fei LI()

Key Laboratory of Aerospace Information Perception and Photoelectric Control, Ministry of Education, School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710129, China

Received:2016-10-19 Online:2018-02-26 Published:2018-02-23
Contact: Xiaoguang GAO E-mail:wankaifang@nwpu.edu.cn;cxg2012@nwpu.edu.cn;libo803@nwpu.edu.cn;nwpulf@mail.nwpu.edu.cn
About author:WAN Kaifang was born in 1987. He received his B.E. in degree detection homing and control technology from Northwestern Polytechnical University (NPU), Xi'an in 2010. He was awarded with an admission from B.E. to Ph.D. directly in 2010 and he was awarded with Ph.D degree in system engineering in 2016. And now he is an assistant researcher of the Key Laboratory of Aerospace Information Perception and Photoelectric Control, Ministry of Education, NPU. His current research interests include sensor management application, multi-agent theory, approximate dynamic programming and reinforcement learning theory. E-mail: wankaifang@nwpu.edu.cn|GAO Xiaoguang was born in 1957. She received her B.E. degree in detection homing and control technology from NPU in 1982. She completed her master degree in system engineering from NPU in 1986. She was awarded with Ph.D. degree from NPU in 1989. She is currently a professor and the head in the Key Laboratory of Aerospace Information Perception and Photoelectric Control, Ministry of Education, NPU. Her research interests are machine learning theory, Bayesian network theory, and multi-agent control application. E-mail: cxg2012@nwpu.edu.cn|LI Bo was born in 1978. He received his B.E. degree in detection homing and control technology from NPU in 2000. He obtained his master degree in system engineering from NPU in 2003. He was awarded with Ph.D. degree from NPU in 2008. He is an associate professor of NPU now. Her research interests are machine learning theory, and multi-agent control application. E-mail: libo803@nwpu.edu.cn|LI Fei was born in 1988. He received his B.E. degree in detection homing and control technology from NPU, Xi'an in 2010. He obtained his master degree in 2012 and he has been working for his Ph.D. degree in system engineering, NPU, since 2012. His current research interests include machine learning theory and deep learning theory. E-mail: nwpulf@mail.nwpu.edu.cn
Supported by:
the National Natural Science Foundation of China(61573285);the National Natural Science Foundation of China(61305133);This work was supported by the National Natural Science Foundation of China (61573285; 61305133)

Abstract

Abstract:

This paper researches the adaptive scheduling problem of multiple electronic support measures (multi-ESM) in a ground moving radar targets tracking application. It is a sequential decision-making problem in uncertain environment. For adaptive selection of appropriate ESMs, we generalize an approximate dynamic programming (ADP) framework to the dynamic case. We define the environment model and agent model, respectively. To handle the partially observable challenge, we apply the unsented Kalman filter (UKF) algorithm for belief state estimation. To reduce the computational burden, a simulation-based approach rollout with a redesigned base policy is proposed to approximate the long-term cumulative reward. Meanwhile, Monte Carlo sampling is combined into the rollout to estimate the expectation of the rewards. The experiments indicate that our method outperforms other strategies due to its better performance in larger-scale problems.

Key words: sensor scheduling, target tracking, approximate dynamic programming, non-myopic, rollout, belief state

Kaifang WAN, Xiaoguang GAO, Bo LI, Fei LI. Using approximate dynamic programming for multi-ESM scheduling to track ground moving targets[J]. Journal of Systems Engineering and Electronics, 2018, 29(1): 74-85.

Figures/Tables 5

References 26

1	HE Y, CHONG E K P. Sensor scheduling for target tracking: a Monte Carlo sampling approach. Digital Signal Processing, 2006, 16 (5): 533- 545. doi: 10.1016/j.dsp.2005.02.005
2	SCOTT A M, ZACHARY A H, CHONG E K P. A POMDP framework for coordinated guidance of autonomous UAVs for multitarget tracking. EURASIP Journal on Advances in Signal Processing, 2009, 2009 (1): 1- 17.
3	YAO P, WANG H, SU Z. Real-time path planning of unmanned aerial vehicle for target tracking and obstacle avoidance in complex dynamic environment. Aerospace Science & Technology, 2015, 47, 269- 279.
4	SHAFERMAN V, SHIMA T. Tracking multiple ground targets in urban environment using cooperating unmanned aerial vehicles. Journal of Dynamic Systems Management & Control, 2015, 137 (5): 1- 11.
5	ZHANG M, LIU H. Cooperative tracking a moving target using multiple fixed-wing UAVs. Journal of Intelligent & Robotic System, 2016, 81 (3): 505- 529.
6	HERNANDEZ M L, KIRUBARAJAN T, BAR-SHALOM Y. Multisensor resource deployment using posterior Cramer-Rao bounds. IEEE Trans. on Aerospace and Electronic Systems, 2004, 40 (2): 399- 416. doi: 10.1109/TAES.2004.1309993
7	LI B B, KIUREGHIAN A D. Robust optimal sensor placement for operational model analysis based on maximum expected unity. Mechanical System and Signal Processing, 2016, 75, 155- 175. doi: 10.1016/j.ymssp.2016.01.005
8	GONG X, ZHANG J, COCHRAN D. Optimal placement for barrier coverage in bistatic radar sensor networks. IEEE/ACM Trans. on Networking, 2016, 24 (1): 259- 271. doi: 10.1109/TNET.2014.2360849
9	LIU X, SHAN G, SHI J. Multi-sensor allocation based on Cramer-Rao low bound. Proc. of IEEE Workshop on Electronics, Computer and Applications, 2014: 467-470.
10	VERMA A, AKELLA M, FREEZE J. Sensor resource management for sub-orbital multi-target tracking and discrimination. Proc. of AIAA Space Flight Mechanics Meeting, 2015: 1647-1662.
11	GOSTAR A K, HOSEINNEZHAD R, BAB-HADIASHAR A. Robust multi-Bernoulli sensor selection for multi-target tracking in sensor networks. IEEE Signal Processing Letters, 2014, 20 (12): 1167- 1170.
12	ZOGHI M, KAHAEI M H. Adaptive sensor selection in wireless sensor networks for target tracking. IET Signal Processing, 2010, 4 (5): 530- 536. doi: 10.1049/iet-spr.2008.0205
13	BEJAR R, KRISHNAMACHARI B, GOMES C. Distributed constraint satisfaction in a wireless sensor tracking system. Proc. of the Workshop on Distributed Constraint Reasoning, 2001: 81-90.
14	HORRIDGE P R, HERNANDEZ M L. Performance bounds for angle-only filtering with application to sensor network management. Proc. of the 6th International Conference on Information Fusion, 2003: 695-703.
15	HAUNG C F. The coverage problem in a wireless sensor network. Proc. of the 2nd ACM International Conference on Wireless Sensor Networks and Applications, 2003: 115-121.
16	GUPTA V, CHUNG T H. On a stochastic selection algorithm with applications in sensor scheduling and sensor coverage. Automatica, 2006, 42 (2): 251- 260. doi: 10.1016/j.automatica.2005.09.016
17	THARMARASA R. Sensor management for large-scale multisensor-multitarget tracking. Hamilton, Canada: McMaster University, 2007.
18	RISTIC B, VO B N, CLARK D. A note on the reward function for RHD filter with sensor control. IEEE Trans. on Aerospace & Electronic Systems, 2011, 47 (2): 1521- 1529.
19	HOANG H G. Control of a mobile sensor for multi-target tracking using multi-target /object multi-Bernoulli filter. Proc. of International Conference on Control, Automation and Information Sciences, 2012: 7-12.
20	DARIN C H. Stochastic control approaches for sensor management in search and exploitation. Massachusetts, USA: Boston University, 2010.
21	WARREN B P. Approximate dynamic programming: solving the curses of dimensionality. New Jersey: Wiley Press, 2010.
22	CHRIS K, DORON B, ALFRED H. Adaptive multi-modality sensor scheduling for detection and tracking of smart targets. Digital Signal Processing, 2006, 16 (5): 546- 567. doi: 10.1016/j.dsp.2004.12.008
23	BAR-SHALOM Y, LI X, KIRUBARAJAN T. Estimation with applications to tracking and navigation. New Jersey: Wiley Press, 2011.
24	LI Y, KRAKOW L W. Approximate stochastic dynamic programming for sensor scheduling to track multiple targets. Digital Signal Processing, 2009, 19 (6): 978- 989. doi: 10.1016/j.dsp.2007.05.004
25	BERTSEKAS D P, TSITSIKLIS J N. Neuro-dynamic programming. Massachusetts: Athena Scientific Press, 1996.
26	SHAWE-TAYLOR J, CRISTIANINI N. Kernel methods for pattern analysis. Journal of the American Statistical Association, 2006, 101 (476): 1730.

[1]	Zhuanhua ZHANG, Gongjian ZHOU. Maneuvering target state estimation based on separate modeling of target trajectory shape and dynamic characteristics [J]. Journal of Systems Engineering and Electronics, 2022, 33(5): 1195-1209.
[2]	Keyi LI, Zhengkun GUO, Gongjian ZHOU. State estimation in range coordinate using range-only measurements [J]. Journal of Systems Engineering and Electronics, 2022, 33(3): 497-510.
[3]	Yuanshi ZHANG, Minghai PAN, Weijun LONG, Hua LI, Qinghua HAN. Joint waveform selection and power allocation algorithm in manned/unmanned aerial vehicle hybrid swarm based on chance-constraint programming [J]. Journal of Systems Engineering and Electronics, 2022, 33(3): 551-562.
[4]	Chen TIAN, Yang PEI, Peng HOU, Qian ZHAO. Multi-target tracking algorithm based on PHD filter against multi-range-false-target jamming [J]. Journal of Systems Engineering and Electronics, 2020, 31(5): 859-870.
[5]	Yang SU, Ting CHENG, Zishu HE, Xi LI, Yanxi LU. Adaptive resource management for multi-target tracking in co-located MIMO radar based on time-space joint allocation [J]. Journal of Systems Engineering and Electronics, 2020, 31(5): 916-927.
[6]	Zhichao BAO, Qiuxi JIANG, Fangzheng LIU. Multiple model efficient particle filter based track-before-detect for maneuvering weak targets [J]. Journal of Systems Engineering and Electronics, 2020, 31(4): 647-656.
[7]	Gongguo XU, Ganlin SHAN, Xiusheng DUAN. Sensor scheduling for ground maneuvering target tracking in presence of detection blind zone [J]. Journal of Systems Engineering and Electronics, 2020, 31(4): 692-702.
[8]	Lili SUN, Yunhe CAO, Wenhua WU, Yutao LIU. A multi-target tracking algorithm based on Gaussian mixture model [J]. Journal of Systems Engineering and Electronics, 2020, 31(3): 482-487.
[9]	Haowei ZHANG, Junwei XIE, Jiaang GE, Zhaojian ZHANG, Wenlong LU. Finite sensor selection algorithm in distributed MIMO radar for joint target tracking and detection [J]. Journal of Systems Engineering and Electronics, 2020, 31(2): 290-302.
[10]	Haowei ZHANG, Junwei XIE, Junpeng SHI, Zhaojian ZHANG. Antenna selection in MIMO radar with collocated antennas [J]. Journal of Systems Engineering and Electronics, 2019, 30(6): 1119-1131.
[11]	Mahmoudreza HADAEGH, Hamid KHALOOZADEH, Mohammadtaghi BEHESHTI. Augmented input estimation in multiple maneuvering target tracking [J]. Journal of Systems Engineering and Electronics, 2019, 30(5): 841-851.
[12]	Hongwei ZHANG, Weixin XIE. Constrained auxiliary particle filtering for bearings-only maneuvering target tracking [J]. Journal of Systems Engineering and Electronics, 2019, 30(4): 684-695.
[13]	Na'e ZHENG, Yang SUN, Xiyu SONG, Song CHEN. Joint resource allocation scheme for target tracking in distributed MIMO radar systems [J]. Journal of Systems Engineering and Electronics, 2019, 30(4): 709-719.
[14]	Qiming YANG, Jiandong ZHANG, Guoqing SHI. Modeling of UAV path planning based on IMM under POMDP framework [J]. Journal of Systems Engineering and Electronics, 2019, 30(3): 545-554.
[15]	Zheng QIN, Yan'gang LIANG. Sensor management of LEO constellation based on covariance control [J]. Journal of Systems Engineering and Electronics, 2019, 30(2): 393-401.

Using approximate dynamic programming for multi-ESM scheduling to track ground moving targets

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 5

References 26

Related Articles 15

Recommended Articles

Metrics

Comments