User space transformation in deep learning based recommendation

doi:10.23919/JSEE.2020.000043

Journal of Systems Engineering and Electronics ›› 2020, Vol. 31 ›› Issue (4): 674-684.doi: 10.23919/JSEE.2020.000043

• Electronics Technology • Previous Articles Next Articles

User space transformation in deep learning based recommendation

Caihua WU*(), Jianchao MA(), Xiuwei ZHANG(), Dang XIE()

Radar Non-Commissioned Officer School, Air Force Early Warning Academy, Wuhan 430345, China

Received:2019-03-15 Online:2020-08-25 Published:2020-08-25
Contact: Caihua WU E-mail:wucaihua2009@163.com;549464509@qq.com;39353745@qq.com;9972653@qq.com
About author:WU Caihua was born in 1980. She received her B.S. and M.S. degrees in computer application from Ordnance Engineering College in 2003 and 2006 respectively, and Ph.D. degree in weapon system and application from Ordnance Engineering College in 2009. She is currently a lecturer in Air Force Early Warning Academy. Her research interests include machine learning, software reliability and information system. E-mail: wucaihua2009@163.com|MA Jianchao was born in 1972. He received his B.S. degree in command automation engineering from Air Force Radar Academy in 1994, and M.S. degree in military intelligence from Air Force Radar Academy in 1997. He is currently an associate professor in Air Force Early Warning Academy. His research interests include machine learning and information system. E-mail: 549464509@qq.com|ZHANG Xiuwei was born in 1980. He received his B.S. degree in computer science and technology from Air Force Radar Academy in 2001, and M.S. degree in computer application from Wuhan University of Technology in 2009. He received his Ph.D. degree in computer software and theory from Wuhan University in 2014. He is currently a lecturer in Air Force Early Warning Academy. His research interests include machine learning and information system. E-mail: 39353745@qq.com|XIE Dang was born in 1979. He received his B.S. degree in information and computing science from Hangzhou University of Electronic Science and Technology in 2003, and M.S. degree in information and signal processing from Air Force Radar Academy in 2009. He is currently a lecturer in Air Force Early Warning Academy. His research interests include machine learning and information system. E-mail: 9972653@qq.com
Supported by:
the National Natural Science Foundation of China(61403350);This work was supported by the National Natural Science Foundation of China (61403350)

Abstract

Abstract:

Deep learning based recommendation methods, such as the recurrent neural network based recommendation method (RNNRec) and the gated recurrent unit (GRU) based recommendation method (GRURec), are proposed to solve the problem of time heterogeneous feedback recommendation. These methods out-perform several state-of-the-art methods. However, in RNNRec and GRURec, action vectors and item vectors are shared among users. The different meanings of the same action for different users are not considered. Similarly, different user preference for the same item is also ignored. To address this problem, the models of RNNRec and GRURec are modified in this paper. In the proposed methods, action vectors and item vectors are transformed into the user space for each user firstly, and then the transformed vectors are fed into the original neural networks of RNNRec and GRURec. The transformed action vectors and item vectors represent the user specified meaning of actions and the preference for items, which makes the proposed method obtain more accurate recommendation results. The experimental results on two real-life datasets indicate that the proposed method outperforms RNNRec and GRURec as well as other state-of-the-art approaches in most cases.

Key words: recommender system, collaborative filtering, time heterogeneous feedback, recurrent neural network, gated recurrent unit (GRU), user space transformation

Caihua WU, Jianchao MA, Xiuwei ZHANG, Dang XIE. User space transformation in deep learning based recommendation[J]. Journal of Systems Engineering and Electronics, 2020, 31(4): 674-684.

Figures/Tables 10

Fig 1

Fig 2

Fig 3

Table 1

Table 2

Table 3

Table 4

Table 5

Fig 4

Fig 5

References 27

1	WU C H, WANG J W, LIU J T, et al. Recurrent neural network based recommendation for time heterogeneous feedback. Knowledge-Based Systems, 2016, 109, 90- 103. doi: 10.1016/j.knosys.2016.06.028
2	LIU J T, WU C H, WANG J W. Gated recurrent units based neural network for time heterogeneous feedback recommendation. Information Sciences, 2018, 423, 50- 65. doi: 10.1016/j.ins.2017.09.048
3	LIU J T, WU C H. Deep learning based recommendation: a survey. Proc. of the International Conference on Information Science and Applications, 2017: 451-458.
4	ZHANG S, YAO L, SUN A. Deep learning based recommender system: a survey and new perspectives. http://arXiv.org/abs/1707.07435.
5	SALAKHUTDINOV R, MNIH A, REYHINTON G. Restricted Boltzmann machines for collaborative filtering. Proc. of the 24th International Conference on Machine Learning, 2007: 791-798.
6	OORDA V D, DIELEMAN S, SCHRAUWEN B. Deep content-based music recommendation. Proc. of the Annual Conference on Neural Information Processing Systems, 2013: 2643-2651.
7	WANG H, WANG N, YEUNG D Y. Collaborative deep learning for recommender systems. Proc. of the 21st International Conference on Knowledge Discovery and Data Mining, 2015: 1235-1244.
8	VINCENT P, LAROCHELLE H, LAJOIE I, et al. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research, 2010, 11 (12): 3371- 3408.
9	WANG C, BLEI D M. Collaborative topic modeling for recommending scientific articles. Proc. of the 17th International Conference on Knowledge Discovery and Data Mining, 2011: 448-456.
10	ELKAHKY A M, SONG Y, HE X. A multi-view deep learning approach for cross domain user modeling in recommendation systems. Proc. of the 24th International Conference on World Wide Web, 2015: 278-288.
11	WANG J, YU L, ZHANG W, et al. Irgan: a minimax game for unifying generative and discriminative information retrieval models. Proc. of the 40th International Conference on Research and Development in Information Retrieval, 2017: 515-524.
12	ZHANG J, CAI H, HUANG T, et al. Distributional representation model for collaborative filtering. http://arXiv.org/abs/1502.04163.
13	HIDASI B, KARATZOGLOU A, BALTRUNAS L, et al. Session-based recommendations with recurrent neural networks. http://arXiv.org/abs/1511.06939.
14	EBESU T, SHEN B, FANG Y. Collaborative memory network for recommendation systems. Proc. of the Special Interest Group on Information Retrieval, 2018: 515-524.
15	LI X P, SHE J. Collaborative variational autoencoder for recommender systems. Proc. of the 23th ACM SIGKDD Conference Knowledge Discovery and Data Mining, 2017: 305-314.
16	LIANG D, KRISHNAN R G, HOFFMANM D, et al. Variational autoencoders for collaborative filtering. Proc. of the 27th World Wide Web Conference, 2018: 689-698.
17	CHUNG J, GULCEHRE C, CHO K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. http://arXiv.org/abs/1412.3555.
18	CHO K, MERRIENBOER B V, GULCEHRE C, et al. Learning phrase representations using RNN encoder decoder for statistical machine translation. http://arXiv.org/abs/1406.1078.
19	HINTON G, SRIVASTAVA N, SWERSKY K. Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning. http://www.cs.toronto.edu/tijmen/csc321/slides/lectureslideslec6.pdf.
20	CANTADOR I, BRUSILOVSKY P, KUFLIK T. Second workshop on information heterogeneity and fusion in recommender systems. Proc. of the 5th ACM Conference on Recommender Systems, 2011: 387-388.
21	HU Y, KOREN Y, VOLINSKY C. Collaborative filtering for implicit feedback datasets. Proc. of the 8th IEEE International Conference on Data Mining, 2008: 263-272.
22	GANTNER Z, DRUMOND L, FREUDENTHALER C, et al. Bayesian personalized ranking for non-uniformly sampled items. Proc. of the Knowledge Discovery and Data Mining Cup and Workshop, 2011: 231-247.
23	RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: Bayesian personalized ranking from implicit feedback. Proc. of the 25th Conference on Uncertainty in Artificial Intelligence, 2009: 452-461.
24	WEIMER M, KARATZOHLOU A, SMOLA A. Improving maximum-margin matrix factorization. Machine Learning, 2008, 72 (3): 263- 276. doi: 10.1007/s10994-008-5073-7
25	SREBRO N, RENNIE J D M, JAAKKOLA T S. Maximum-margin matrix factorization. Proc. of the Annual Conference on Neural Information Processing Systems, 2005: 1329-1336.
26	GANTNER Z, RENDLE S, FREUDENTHALER C, et al. Mymedialite: a free recommender system library. Proc. of the 5th ACM Conference on Recommender Systems, 2011: 305-308.
27	ORAMAS S, OSTUNI V C, NOIAT D, et al. Sound and music recommendation with knowledge graphs. ACM Trans. on Intelligent Systems and Technology, 2016, 8 (2): 1- 21.

Data set	Taobao 2014	Lastfm
Number of users	884	1 892
Number of items	9 531	17 632
Number of feedback	182 880	186 479
Average feedback per user	206.88	98.57
Average feedback per item	19.18	10.58
Density	2.17$\times$10$^{ - 2}$	5.59$\times$10$^{ - 3}$
Kinds of feedback	4	1

Method	$D=16$		$D=32$
Method	F1@10	F1@20	F1@10	F1@20
KNN(user)	0.026 0(0.000 0)	0.025 8(0.000 0)	0.026 0(0.000 0)	0.025 8(0.000 0)
KNN(item)	0.021 4(0.000 0)	0.022 3(0.000 0)	0.021 4(0.000 0)	0.022 3(0.000 0)
Most popular	0.024 8(0.000 0)	0.025 7(0.000 0)	0.024 8(0.000 0)	0.025 7(0.000 0)
WRMF	0.030 8(0.000 5)	0.029 5(0.000 4)	0.029 8(0.000 7)	0.028 0(0.000 4})
BPRMF	0.017 1(0.001 3)	0.018 2(0.001 3)	0.017 5(0.001 5)	0.008 4(0.001 0)
Weighted BPRMF	0.007 7(0.001 3)	0.007 8(0.000 6)	0.007 6(0.001 4)	0.008 4(0.000 9)
SoftmarginRankingMF	0.009 8(0.001 8)	0.011 6(0.001 6)	0.011 1(0.001 7)	0.011 7(0.000 9)
RNNRec	0.048 1(0.002 7)	0.048 2(0.002 2)	0.045 8(0.002 2)	0.046 6(0.002 0)
TransRNNRec	0.062 3(0.003 3)	0.058 3(0.001 5)	0.066 9(0.001 7)	0.059 4(0.001 5)
GRURec	0.051 4(0.000 8)	0.048 8(0.000 9)	0.051 9(0.002 0)	0.049 7(0.001 1)
TransGRURec	0.063 3(0.001 8)	0.058 0(0.002 0)	0.065 8(0.001 9)	0.059 3(0.001 3)

Method	$D=16$		$D=32$
Method	F1@10	F1@20	F1@10	F1@20
KNN(user)	0.036 5(0.000 0)	0.038 9(0.000 0)	0.036 5(0.000 0)	0.038 9(0.000 0)
KNN(item)	0.028 3(0.000 0)	0.032 1(0.000 0)	0.028 3(0.000 0)	0.032 1(0.000 0)
Most popular	0.032 6(0.000 0)	0.038 0(0.000 0)	0.032 6(0.000 0)	0.038 0(0.000 0)
WRMF	0.039 3(0.000 9)	0.044 3(0.000 3)	0.035 4(0.001 2)	0.041 0(0.000 9)
BPRMF	0.022 1(0.001 3)	0.027 1(0.001 2)	0.025 8(0.001 2)	0.029 6(0.001 4)
Weighted BPRMF	0.009 0(0.000 4)	0.011 2(0.000 6)	0.010 4(0.001 1)	0.029 6(0.001 4)
SoftmarginRankingMF	0.013 7(0.001 5)	0.017 9(0.001 7)	0.014 6(0.002 7)	0.018 5(0.002 0)
RNNRec	0.057 6(0.002 1)	0.064 1(0.001 6)	0.057 3(0.002 6)	0.063 7(0.002 1)
TransRNNRec	0.072 2(0.000 5)	0.074 5(0.001 5)	0.076 4(0.001 3)	0.076 8(0.000 7)
GRURec	0.061 0(0.001 2)	0.068 0(0.000 9)	0.062 6(0.001 8)	0.068 0(0.001 8)
TransGRURec	0.073 2(0.001 0)	0.075 8(0.001 0)	0.074 4(0.001 7)	0.075 7(0.001 9)

Method	$D=16$		$D=32$
Method	F1@10	F1@20	F1@10	F1@20
KNN(user)	0.023 1(0.000 0)	0.021 4(0.000 0)	0.023 1(0.000 0)	0.021 4(0.000 0)
KNN(item)	0.025 5(0.000 0)	0.024 5(0.000 0)	0.025 5(0.000 0)	0.024 5(0.000 0)
Most popular	0.013 9(0.000 0)	0.011 9(0.000 0)	0.013 9(0.000 0)	0.011 9(0.000 0)
WRMF	0.023 1(0.000 2)	0.023 1(0.000 2)	0.023 3(0.000 2)	0.023 0(0.000 5)
BPRMF	0.017 5(0.001 3)	0.017 1(0.000 6)	0.018 5(0.000 7)	0.017 5(0.000 4)
Weighted BPRMF	0.017 7(0.001 1)	0.017 7(0.000 7)	0.018 8(0.001 4)	0.018 8(0.001 4)
SoftmarginRankingMF	0.010 6(0.001 7)	0.011 3(0.001 5)	0.009 7(0.001 1)	0.010 4(0.001 1)
RNNRec	0.055 0(0.001 5)	0.037 4(0.001 1)	0.070 5(0.001 5)	0.047 8(0.001 3)
TransRNNRec	0.077 9(0.003 5)	0.050 6(0.001 3)	0.094 5(0.001 5)	0.057 0(0.001 2)
GRURec	0.079 8(0.002 5)	0.055 8(0.000 7)	0.111 6(0.001 6)	0.074 7(0.001 7)
TransGRURec	0.054 0(0.003 4)	0.040 3(0.001 9)	0.089 0(0.005 7)	0.056 0(0.002 6)

Method	$D=16$		$D=32$
Method	F1@10	F1@20	F1@10	F1@20
KNN(user)	0.030 9(0.000 0)	0.033 1(0.000 0)	0.030 9(0.000 0)	0.033 1(0.000 0)
KNN(item)	0.032 8(0.000 0)	0.035 7(0.000 0)	0.032 8(0.000 0)	0.035 7(0.000 0)
Most popular	0.016 7(0.000 0)	0.018 5(0.000 0)	0.016 7(0.000 0)	0.018 5(0.000 0)
WRMF	0.033 9(0.000 3)	0.036 9(0.000 3)	0.034 6(0.000 4)	0.037 8(0.000 3)
BPRMF	0.024 8(0.001 3)	0.027 2(0.001 1)	0.027 6(0.000 7)	0.029 9(0.001 3)
Weighted BPRMF	0.024 8(0.000 9)	0.027 2(0.001 1)	0.027 6(0.000 7)	0.029 9(0.001 3)
SoftmarginRankingMF	0.014 4(0.001 3)	0.017 1(0.001 4)	0.013 4(0.001 8)	0.017 5(0.000 9)
RNNRec	0.056 7(0.000 8)	0.043 7(0.000 9)	0.078 0(0.002 2)	0.059 0(0.002 7)
TransRNNRec	0.082 5(0.001 5)	0.056 8(0.000 4)	0.095 7(0.001 3)	0.062 8(0.000 7)
GRURec	0.095 7(0.002 9)	0.071 6(0.002 4)	0.114 2(0.001 4)	0.085 3(0.000 7)
TransGRURec	0.051 4(0.003 6)	0.041 9(0.002 0)	0.076 8(0.005 1)	0.054 7(0.003 5)

User space transformation in deep learning based recommendation

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 10

References 27

Related Articles 2

Recommended Articles

Metrics

Comments

[1]	Jiandong ZHANG, Qiming YANG, Guoqing SHI, Yi LU, Yong WU. UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning [J]. Journal of Systems Engineering and Electronics, 2021, 32(6): 1421-1438.
[2]	Ma Haibo, Zhang Liguo & Chen Yangzhou. Recurrent neural network for vehicle dead-reckoning [J]. Journal of Systems Engineering and Electronics, 2008, 19(2): 351-355.