Multi-agent system application in accordance with game theory in bi-directional coordination network model

doi:10.23919/JSEE.2020.000006

Journal of Systems Engineering and Electronics ›› 2020, Vol. 31 ›› Issue (2): 279-289.doi: 10.23919/JSEE.2020.000006

• Electronics Technology • Previous Articles Next Articles

Multi-agent system application in accordance with game theory in bi-directional coordination network model

Jie ZHANG^1,*(), Gang WANG²(), Shaohua YUE²(), Yafei SONG²(), Jiayi LIU²(), Xiaoqiang YAO²()

¹ College of Electronics and Information Engineering, Air Force Engineering University, Xi'an 710054, China
² College of Air Missile Defense, Air Force Engineering University, Xi'an 710054, China

Received:2019-04-30 Online:2020-04-30 Published:2020-04-30
Contact: Jie ZHANG E-mail:afeu_zhangjie@163.com;sharesunny123@163.com;zhouguoan@sina.cn;yafei_song@163.com;sixandone1@163.com;icemissile@sina.com
About author:ZHANG Jie was born in 1995. He is a master degree candidate at the Air Force Engineering University. His research interests are combat multi-agent based on deep learning and tactical air defense and antimissile command and control system.E-mail: afeu_zhangjie@163.com|WANG Gang was born in 1975. He received his Ph.D. degree from the Air Force Engineering University. His research interests are machine learning, information fusion and command and control system. E-mail: sharesunny123@163.com|YUE Shaohua was born in 1968. She received her Ph.D. degree from the Air Force Engineering University. Her research interests are command information system and intelligent command and control. E-mail: zhouguoan@sina.cn|SONG Yafei was born in 1988. He received his Ph.D. degree from the Air Force Engineering University. His research interests are pattern recognition and intelligent information processing. E-mail: yafei_song@163.com|LIU Jiayi was born in 1996. He is a master degree candidate at the Air Force Engineering University. His research interests are air defense and anti-missile command and control system and intelligent decision-making based on reinforcement learning. E-mail: sixandone1@163.com|YAO Xiaoqiang was born in 1985. He received his Ph.D. degree from the Air Force Engineering University. His research interests are intelligent information processing and simulation training and simulation.E-mail: icemissile@sina.com
Supported by:
the National Natural Science Foundation of China(61503407);the National Natural Science Foundation of China(61806219);the National Natural Science Foundation of China(61703426);the National Natural Science Foundation of China(61876189);the National Natural Science Foundation of China(61703412);the China Postdoctoral Science Foundation(2016 M602996);This work was supported by the National Natural Science Foundation of China (61503407; 61806219; 61703426; 61876189; 61703412) and the China Postdoctoral Science Foundation (2016 M602996)

Abstract

Abstract:

The multi-agent system is the optimal solution to complex intelligent problems. In accordance with the game theory, the concept of loyalty is introduced to analyze the relationship between agents' individual income and global benefits and build the logical architecture of the multi-agent system. Besides, to verify the feasibility of the method, the cyclic neural network is optimized, the bi-directional coordination network is built as the training network for deep learning, and specific training scenes are simulated as the training background. After a certain number of training iterations, the model can learn simple strategies autonomously. Also, as the training time increases, the complexity of learning strategies rises gradually. Strategies such as obstacle avoidance, firepower distribution and collaborative cover are adopted to demonstrate the achievability of the model. The model is verified to be realizable by the examples of obstacle avoidance, fire distribution and cooperative cover. Under the same resource background, the model exhibits better convergence than other deep learning training networks, and it is not easy to fall into the local endless loop. Furthermore, the ability of the learning strategy is stronger than that of the training model based on rules, which is of great practical values.

Key words: loyalty, game theory, bi-directional coordination network, multi-agent system, learning strategy

Jie ZHANG, Gang WANG, Shaohua YUE, Yafei SONG, Jiayi LIU, Xiaoqiang YAO. Multi-agent system application in accordance with game theory in bi-directional coordination network model[J]. Journal of Systems Engineering and Electronics, 2020, 31(2): 279-289.

Figures/Tables 20

Fig 1

Fig 2

Fig 3

Fig 4

Fig 5

Fig 6

Fig 7

Fig 8

Table 1

Fig 9

Fig 10

Fig 11

Fig 12

Fig 13

Fig 14

Fig 15

Table 2

Fig 16

Fig 17

Table 3

References 36

1	CASE D O. The society of mind. Information Processing & Management, 1988, 24 (4): 499- 500.
2	HU J P, HONG Y G. Leader-following coordination of multi-agent systems with coupling time delays. Physica A:Statistical Mechanics and its Applications, 2007, 374 (2): 853- 863. doi: 10.1016/j.physa.2006.08.015
3	GUPTA J K, EGOROV M, KOCHENDERFER M.Coopera-linebreak tive multi-agent control using deep reinforcement learning.Proc.of the International Conference on Autonomous Agents and Multiagent Systems, 2017: 66-83.
4	BÚRDALO L, TERRASA A, JULIÁN V, et al. The information flow problem in multi-agent systems. Engineering Applications of Artificial Intelligence, 2018, 70, 130- 141. doi: 10.1016/j.engappai.2018.01.011
5	KAYA M, ALHAJJ R.Modular fuzzy-reinforcement learning approach with internal model capabilities for multi-agent systems.IEEE Trans.on Systems Man & Cybernetics Part B Cybernetics A Publication of the IEEE Systems Man & Cybernetics Society, 2004, 34(2): 1210-1223.
6	ALLOUCHE M K, BOUKHTOUTA A. Multi-agent coordination by temporal plan fusion:application to combat search and rescue. Information Fusion, 2010, 3 (11): 220- 232.
7	SUN X, LIU Y, LI J, et al. Feature evaluation and selection with cooperative game theory. Pattern Recognition, 2012, 8 (45): 2992- 3002.
8	RAHWAN T, MICHALAK T, WOOLDRIDGE M, et al. Anytime coalition structure generation in multi-agent systems with positive or negative externalities. Artificial Intelligence, 2012, 186, 95- 122. doi: 10.1016/j.artint.2012.03.007
9	XU X H, WANG Y, LIU J, et al.Analysis on the achievement milestones and limitations of Game Theory.Proc.of the Control & Decision Conference, 2008: 1214-1219.
10	FARRELL J, RABIN M. Cheap talk. Journal of Economic Perspectives, 1996, 10 (3): 103- 118.
11	KAGEL J H, ROTH A E. The handbook of experimental economics. Princeton: Princeton University Press, 2016.
12	OISHI S, FUKUTA N.A cooperative task execution mecha-linebreak nism for personal assistant agents using ability onto-linebreak logy.Proc.of the IEEE/WIC/ACM International Conference on Web Intelligence, 2016: 664-667.
13	GITINAVARD H, MOUSAVI S M, VAHDANI B. A new multi-criteria weighting and ranking model for group decision-making analysis based on interval-valued hesitant fuzzy sets to selection problems. Neural Computing & Applications, 2016, 27 (6): 1593- 1605.
14	TOHMÉF, SANDHOLM T. Coalition formation processes with belief revision among bounded-rational self-interested agents. Journal of Logic & Computation, 2018, 9 (6): 793- 815.
15	CǍTǍLIN D, ENEA C, GUELEV D. Model-checking an alternating-time temporal logic with knowledge, imperfect information, perfect recall and communicating coalitions. Electronic Proceedings in Theoretical Computer Science, 2018, 25, 103- 117.
16	LU Z H, ZHANG L, WANG L. Controllability analysis of multi-agent systems with switching topology over finite fields. Science China (Information Sciences), 2019, 62 (1): 1- 15.
17	WEN G, CHEN C L P, LIU Y J, et al. Neural network-based adaptive leader-following consensus control for a class of nonlinear multiagent state-delay systems. IEEE Trans.on Cybernetics, 2016, 47 (8): 2151- 2160.
18	KUNITO G, AIZAWA K, HATORI M. Tracking agent for communication between multiple cooperative agents. Electronics & Communications in Japan, 2001, 84 (5): 11- 20.
19	DOU C, YUE D, HAN Q L, et al. A multi-agent system based event-triggered hybrid control scheme for energy internet. IEEE Access, 2017, 99 (5): 3263- 3272.
20	LEI Z, WEI G, YAN D W, et al. Study of reconfiguration for the distribution network with distributed generations based on multi-agent alliance algorithm. Power System Protection & Control, 2012, 40 (10): 95- 101.
21	CICIRELLI F, GIORDANO A, NIGRO L. Efficient environment management for distributed simulation of large-scale situated multi-agent systems. Concurrency and Computation:Practice and Experience, 2015, 27 (3): 610- 632. doi: 10.1002/cpe.3254
22	YU H, SHEN Z, LEUNG C, et al. A survey of multi-agent trust management systems. IEEE Access, 2013, 1, 35- 50. doi: 10.1109/ACCESS.2013.2259892
23	CAO Y Q, ZHANG Z, HUANG X S, et al. Multi-agent system coalition utility allocation strategy based on loyalty. Computer Science, 2014, 41 (5): 235- 238.
24	LOWE R.Multi-agent actor-critic for mixed cooperative-competitive environments.https://arxiv.org/abs/1706.02275.
25	POTJANS W, MORRISON A, DIESMANN M. A spiking neural network model of an actor-critic learning agent. Neural Computation, 2009, 21 (2): 301- 339.
26	PENG P, WEN Y, YANG Y, et al.Multiagent bidirectionally-coordinated nets: emergence of human-level coordination in learning to play starcraft combat games.https://arxiv.org/abs/1703.10069v4.
27	WEN G, CHEN C L P, FENG J, et al. , Optimized multi-agent formation control based on identifier-actor-critic reinforcement learning algorithm. IEEE Trans.on Fuzzy Systems, 2018, 26 (5): 2719- 2131. doi: 10.1109/TFUZZ.2017.2787561
28	LOWE R, WU Y, TAMAR A, et al.Multi-agent actor-critic for mixed cooperative-competitive environments.https://arxiv.org/abs/1706.02275.
29	YUAN J, LAMPERSKI A.Online control basis selection by a regularized actor critic algorithm.Proc.of the IEEE American Control Conference, 2017: 4448-4453.
30	CHAUDHARI P, SOATTO S.Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks.Proc.of the International Conference on Learning Representations, 2018: 1-10.
31	DING J, ZHU L P, HU B, et al. Stochastic gradient descent based k-means algorithm on large scale data clustering. Applied Mechanics & Materials, 2014, 687/691, 1342- 1345.
32	SONG S, CHAUDHURI K, SARWATE A D.Learning from data with heterogeneous noise using SGD.Proc.of the JMLR Workshop Conference, 2015: 894-902.
33	KUZBORSKIJ I, LAMPERT C H.Data-dependent stability of stochastic gradient descent.https://arxiv.org/abs/linebreak1703.01678v4.
34	ARJOVSKY M, SHAH A, BENGIO Y.Unitary evolution recurrent neural networks.Proc.of the International Conference on Machine Learning, 2015: 1120-1128.
35	WANG C, ZHAO X Z, WANG Y T. Research on decision making method of formation cooperative air defense based on multi-agent cooperation. Communications and Information Processing, 2012, 12 (5): 529- 538.
36	YAN J, GUAN X P, TAN F X. Target tracking and obstacle avoidance for multi-agent systems. International Journal of Automation and Computing, 2010, 7 (4): 550- 556.

Demo_1 000	201	162	0.702 594
Demo_2 000	105	79	0.679 277
Demo_3 000	105	105	0.581 262
Demo_4 000	112	112	0.393 983
Demo_5 000	153	153	0.186 213
Demo_6 000	140	140	0.168 543
Demo_7 000	109	109	0.098 862
Demo_8 000	115	115	0.081 481
Demo_9 000	167	167	0.091 533
Demo_10 000	60	60	0.060 279

Training times	Length of travel route/km	Number of collisions	Variation of fire coverage/%
50	460	7	28
200	490	6	28
500	350	4	27
1 000	330	3	23
2 000	280	1	15
3 000	260	0	15

Training times	Target selection	Attack unit	Consumption of ammunition	Number of feint units	Attack direction selection	Is tactical goal accomplished	Degree of strategy completion/%
200	Radar vehicle: 1	Fighter: 8	Medium-range missile: 10	5	No	No	0
200	Launch vehicle: 1	Fighter: 8	Short-range missile: 0	5	No	No	0
500	Radar vehicle: 1	Fighter: 8	Medium-range missile: 8	5	No	No	0
500	Launch vehicle: 1	Fighter: 8	Short-range missile: 0	5	No	No	0
1 000	Radar vehicle: 1	Fighter: 8	Medium-range missile: 7	5	No	Yes	20
1 000	Launch vehicle: 1	Fighter: 8	Short-range missile: 0	5	No	Yes	20
2 000	Radar vehicle: 1	Fighter: 7	Medium-range missile: 6	6	150$^\circ$	Yes	60
2 000	Launch vehicle: 1	Fighter: 7	Short-range missile: 0	6	150$^\circ$	Yes	60
3 000	Radar vehicle: 1	Fighter: 5	Medium-range missile: 5	4	90$^\circ$	Yes	95
3 000	Launch vehicle: 1	Fighter: 5	Short-range missile: 0	4	90$^\circ$	Yes	95
5 000	Radar vehicle: 1	Fighter: 5	Medium-range missile: 4	3	90$^\circ$;	Yes	98
5 000	Launch vehicle: 1	Fighter: 5	Short-range missile: 1	3	150$^\circ$	Yes	98

[1]	Shouyi LI, Mou CHEN, Qingxian WU, Yuhui WANG. Threat sequencing of multiple UCAVs with incomplete information based on game theory [J]. Journal of Systems Engineering and Electronics, 2022, 33(4): 986-996.
[2]	Sader MALIKA, Fuyong WANG, Zhongxin LIU, Zengqiang CHEN. Distributed fuzzy fault-tolerant consensus of leader-follower multi-agent systems with mismatched uncertainties [J]. Journal of Systems Engineering and Electronics, 2021, 32(5): 1031-1040.
[3]	Duo QI, Junhua HU, Xiaolong LIANG, Jiaqiang ZHANG, Zhihao ZHANG. Research on consensus of multi-agent systems with and without input saturation constraints [J]. Journal of Systems Engineering and Electronics, 2021, 32(4): 947-955.
[4]	Ming ZHANG, Jianjun ZHU, Hehua WANG. Evolutionary game analysis of problem processing mechanism in new collaboration [J]. Journal of Systems Engineering and Electronics, 2021, 32(1): 136-150.
[5]	Bingqiang LI, Tianyi LAN, Yiyun ZHAO, Shuaishuai LYU. Open-loop and closed-loop $D^{\alpha} $ -type iterative learning control for fractional-order linear multi-agent systems with state-delays [J]. Journal of Systems Engineering and Electronics, 2021, 32(1): 197-208.
[6]	Xia WU, Yan LI, Yongjian SUN, Alei CHEN, Jianwen CHEN, Jianchao MA, Hao CHEN. Investigation of MAS structure and intelligent⁺ information processing mechanism of hypersonic target detection and recognition system [J]. Journal of Systems Engineering and Electronics, 2020, 31(6): 1105-1115.
[7]	Weiwei WU, Qian MA, Yexin LIU, Yongjun KIM. A model for knowledge transfer in a multi-agent organization based on lattice kinetic model [J]. Journal of Systems Engineering and Electronics, 2020, 31(1): 156-167.
[8]	Dariush TAVAKOLIFAR, Hamid KHALOOZADEH, Roya AMJADIFARD. Stabilization of switched systems with all unstable modes: application to the aircraft team problem [J]. Journal of Systems Engineering and Electronics, 2019, 30(4): 792-798.
[9]	Zhuoming LI, Xing CHEN, Yu ZHANG, Peng WANG, Wei QIANG, Ningqing LIU. Fuzzy mathematics and game theory based D2D multicast network construction [J]. Journal of Systems Engineering and Electronics, 2019, 30(1): 13-21.
[10]	Chao GENG, Shiyou QU, Yingying XIAO, Mei WANG, Guoqiang SHI, Tingyu LIN, Junjie XUE, Zhengxuan JIA. Diffusion mechanism simulation of cloud manufacturing complex network based on cooperative game theory [J]. Journal of Systems Engineering and Electronics, 2018, 29(2): 321-335.
[11]	Qilong SUN, Naiming QI, Longxu XIAO, Haiqi LIN. Differential game strategy in three-player evasion and pursuit scenarios [J]. Journal of Systems Engineering and Electronics, 2018, 29(2): 352-366.
[12]	Xiaolei Li, Xiaoyuan Luo, Shaobao Li, Jianjin Li, and Xinping Guan. Consensus of second-order nonlinear multi-agent systems via sliding mode observer and controller [J]. Systems Engineering and Electronics, 2017, 28(4): 756-.
[13]	Yanchao Sun, Wenjia Wang, Guangfu Ma, Zhuo Li, and Chuanjiang Li. Backstepping-based distributed coordinated tracking for multiple uncertain Euler-Lagrange systems [J]. Journal of Systems Engineering and Electronics, 2016, 27(5): 1083-1095.
[14]	Yufei Liu, Dechang Pi, and Qiyou Cheng. Ensemble kernel method: SVM classification based on game theory [J]. Systems Engineering and Electronics, 2016, 27(1): 251-.
[15]	Jia Wei and Huajing Fang. Multi-agent consensus with time-varying delays and switching topologies [J]. Journal of Systems Engineering and Electronics, 2014, 25(3): 489-495.

Multi-agent system application in accordance with game theory in bi-directional coordination network model

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 20

References 36

Related Articles 15

Recommended Articles

Metrics

Comments