Constrained voting extreme learning machine and its application

doi:10.23919/JSEE.2021.000018

Journal of Systems Engineering and Electronics ›› 2021, Vol. 32 ›› Issue (1): 209-219.doi: 10.23919/JSEE.2021.000018

• CONTROL THEORY AND APPLICATION • Previous Articles Next Articles

Constrained voting extreme learning machine and its application

Mengcan MIN(), Xiaofang CHEN*(), Yongfang XIE()

¹ School of Automation, Central South University, Changsha 410083, China

Received:2020-02-25 Online:2021-02-25 Published:2021-02-25
Contact: Xiaofang CHEN E-mail:mengcanmin@csu.edu.cn;xiaofangchen@csu.edu.cn;yfxie@csu.edu.cn
About author:|MIN Mengcan was born in 1996. She received her B.E. degree in School of Computer and Information Engineering from Henan University, Kaifeng, China, in 2018. She is currently a postgraduate student at School of Automation, Central South University, Changsha, China. Her research interests include machine learning, knowledge automation, and modeling and control of complex industrial process. E-mail: mengcanmin@csu.edu.cn||CHEN Xiaofang was born in 1975. He received his M.S. degree in control science and engineering and Ph.D. degree in control science and engineering from Central South University, Changsha, China, in 2000 and 2004, respectively. He was with the Baosteel Research Institute and the China North Industries Group, China. Since 2018, he has been a professor with the School of Automation, Central South University. His current research interests include modeling and optimal control of complex industrial process, control and optimization, intelligent control system, and knowledge driven automation.E-mail: xiaofangchen@csu.edu.cn||XIE Yongfang was born in 1972. He received his B.S., M.S. and Ph.D. degrees in control science and engineering from Central South University, Changsha, China, in 1993, 1996 and 1999, respectively. From 1999 to 2003, he did collaborative research in the International Institute of Information Science in Tokyo, Japan and PToPA Institute as a visiting scholar. From 2007 to 2009, he was a visiting scholar at University of Duisburg-Essen, Germany. Currently, he is a full professor in Central South University. His research interests include modeling and control of complex industrial process, decentralized robust control and fault diagnosis. E-mail: yfxie@csu.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61773405; 61751312), and the Major Scientific and Technological Innovation Projects of Shandong Province (2019JZZY020123).

Abstract

Abstract:

Extreme learning machine (ELM) has been proved to be an effective pattern classification and regression learning mechanism by researchers. However, its good performance is based on a large number of hidden layer nodes. With the increase of the nodes in the hidden layers, the computation cost is greatly increased. In this paper, we propose a novel algorithm, named constrained voting extreme learning machine (CV-ELM). Compared with the traditional ELM, the CV-ELM determines the input weight and bias based on the differences of between-class samples. At the same time, to improve the accuracy of the proposed method, the voting selection is introduced. The proposed method is evaluated on public benchmark datasets. The experimental results show that the proposed algorithm is superior to the original ELM algorithm. Further, we apply the CV-ELM to the classification of superheat degree (SD) state in the aluminum electrolysis industry, and the recognition accuracy rate reaches 87.4%, and the experimental results demonstrate that the proposed method is more robust than the existing state-of-the-art identification methods.

Key words: extreme learning machine (ELM), majority voting, ensemble method, sample based learning, superheat degree (SD)

Mengcan MIN, Xiaofang CHEN, Yongfang XIE. Constrained voting extreme learning machine and its application[J]. Journal of Systems Engineering and Electronics, 2021, 32(1): 209-219.

Figures/Tables 17

Fig 1

Fig 2

Fig 3

Fig 4

Fig 5

Fig 6

Table 1

Table 2

Table 3

Fig 7

Table 4

Fig 8

Table 5

Fig 9

Table 6

Fig 10

References 46

1	TAMURA S, TATEISHI M Capabilities of a four-layered feedforward neural network: four layers versus three. IEEE Trans. on Neural Networks, 1997, 8 (2): 251255.
2	WERBOS P. Beyond regression: new tools for prediction and analysis in the behavioral sciences. Cambridge, USA: Harvard University, 1974.
3	MORE J J. Numerical analysis. Heidelberg: Springer, 1978.
4	YAO X Evolving artificial neural networks. Proceedings of the IEEE, 1999, 87 (9): 1423- 1447. doi: 10.1109/5.784219
5	KARNIN E D A simple procedure for pruning back-propagation trained neural networks. IEEE Trans. on Neural Networks, 1990, 1 (2): 239- 242. doi: 10.1109/72.80236
6	HSU C W, LIN C J A comparison of methods for multiclass support vector machines. IEEE Trans. on Neural Networks, 2002, 13 (2): 415- 425. doi: 10.1109/72.991427
7	JIN B, TANG Y C, ZHANG Y Q Support vector machines with genetic fuzzy feature transformation for biomedical data classification. Information Sciences, 2007, 177 (2): 476- 489. doi: 10.1016/j.ins.2006.03.015
8	HUANG G B, ZHU Q Y, SIEW C K Extreme learning machine: theory and applications. Neurocomputing, 2006, 70 (1/3): 489- 501.
9	ZHANG J, DING S F, ZHANG N, et al Incremental extreme learning machine based on deep feature embedded. International Journal of Machine Learning and Cybernetics, 2015, 7 (1): 111- 120.
10	DING S, ZHANG N, ZHANG J, et al Unsupervised extreme learning machine with representational features. International Journal of Machine Learning and Cybernetics, 2015, 8 (2): 587- 595.
11	WANG B Y, ZHAO S, ZHANG S M A distributed load forecasting algorithm based on cloud computing and extreme learning machine. Power System Technology, 2014, 38 (2): 526- 531.
12	ZHU K, YANG H M, MENG K Short-term wind power forecast using extreme learning machine. Journal of Electric Power Science and Technology, 2019, 34 (2): 106- 111.
13	PAN H X, CHENG G J, CAI L Comparison of the extreme learning machine with the support vector machine for reservoir permeability prediction. Computer Engineering & Science, 2010, 32 (2): 131- 134.
14	ZHAO J X, QIAN Y R, NAN F Z, et al The method with CNN multilayer feature fusion and ELM diagnosis for breast diseases. Computer Engineering and Applications, 2020, 56 (4): 122- 127.
15	YU D, DENG L Efficient and effective algorithms for training single-hidden-layer neural networks. Pattern Recognition Letters, 2012, 33 (5): 554- 558. doi: 10.1016/j.patrec.2011.12.002
16	MICHE Y, SORJAMAA A, BAS P, et al OP-ELM: optimally pruned extreme learning machine. IEEE Trans. on Neural Networks, 2010, 21 (1): 158- 162. doi: 10.1109/TNN.2009.2036259
17	LIANG N Y, HUANG G B, SARATCHANDRAN P, et al A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. on Neural Networks, 2006, 17 (6): 1411- 1423. doi: 10.1109/TNN.2006.880583
18	LAN Y, SOH Y C, HUANG G B. A constructive enhancement for online sequential extreme learning machine. Proc. of the International Joint Conference on Neural Networks, 2009. DOI: 10.1109/IJCNN.2009.5178608.
19	MURPHY K P. Machine learning: a probabilistic perspective. Cambridge: The MIT Press, 2012.
20	POLIKAR R Ensemble based systems in decision making. IEEE Circuits and Systems Magazine, 2006, 6 (3): 21- 45. doi: 10.1109/MCAS.2006.1688199
21	HUANG G B, ZHU Q Y, SIEW C K. Extreme learning machine: a new learning scheme of feedforward neural networks. Proc. of the IEEE International Joint Conference on Neural Networks, 2004. DOI: 10.1109/IJCNN.2004.1380068.
22	HUANG G B, CHEN L Enhanced random search based incremental extreme learning machine. Neurocomputing, 2008, 71 (16/18): 3460- 3468.
23	HUANG G B, CHEN L, SIEW C K Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. on Neural Networks, 2006, 17 (4): 879- 892. doi: 10.1109/TNN.2006.875977
24	SU Y, SHAN S G, CHEN X L, et al Classifiability-based discriminatory projection pursuit. IEEE Trans. on Neural Networks, 2011, 22 (12): 2050- 2061. doi: 10.1109/TNN.2011.2170220
25	CAO J, LIN Z, HUANG G B, et al Voting based extreme learning machine. Information Sciences, 2012, 185 (1): 66- 77. doi: 10.1016/j.ins.2011.09.015
26	STOSIC D, STOSIC D, LUDERMIR T Voting based q-generalized extreme learning machine. Neurocomputing, 2016, 174, 1021- 1030. doi: 10.1016/j.neucom.2015.10.028
27	HUANG G B, DING X, ZHOU H Optimization method based extreme learning machine for classification. Neurocomputing, 2010, 74 (1/3): 155- 163.
28	ZHU W, MIAO J, QING L Y. Constrained extreme learning machine: a novel highly discriminative random feedforward neural network. Proc. of the International Joint Conference on Neural Networks, 2014: 800–807.
29	YAN Y T, ZHANG Y P, CHEN J, et al Incomplete data classification with voting based extreme learning machine. Neurocomputing, 2016, 193, 167- 175. doi: 10.1016/j.neucom.2016.01.068
30	HUANG G B, ZHU Q Y, SIEW C K. Extreme learning machine: a new learning scheme of feedforward neural networks. Proc. of the IEEE International Joint Conference on Neural Networks, 2004. DOI: 10.1109/IJCNN.2004.1380068.
31	ZHAO Y, WANG K K Fast cross validation for regularized extreme learning machine. Journal of Systems Engineering and Electronics, 2014, 25 (5): 895- 900. doi: 10.1109/JSEE.2014.000103
32	TANG Y Q, XIAO S Q, ZHAN Y J Predicting settlement along railway due to excavation using empirical method and neural networks. Soils and Foundations, 2019, 59 (4): 1037- 1051. doi: 10.1016/j.sandf.2019.05.007
33	ZHAO Z, CHEN W H, WU X, et al LSTM network: a deep learning approach for short-term traffic forecast. IET Intelligent Transport Systems, 2017, 11 (2): 68- 75. doi: 10.1049/iet-its.2016.0208
34	GUI W H, YUE W C, CHEN X F, et al Process industry knowledge automation and its applications in aluminum electrolysis production. Control Theory and Applications, 2018, 35 (7): 887- 899.
35	CHEN Z G, LI Y G, CHEN X F, et al Semantic network based on intuitionistic fuzzy directed hyper-graphs and application to aluminum electrolysis cell condition identification. IEEE Access, 2017, 5, 20145- 20156. doi: 10.1109/ACCESS.2017.2752200
36	HUANG Z K, YANG C H, CHEN X F, et al Adaptive over-sampling method for classification with application to imbalanced datasets in aluminum electrolysis. Neural Computing and Applications, 2019, 32, 7183- 7199.
37	ZHAO S W, XIE Y F, YUE W C, et al. A machine learning method for state identification of superheat degree with flame interference. Proc. of the 10th International Symposium on High-Temperature Metallurgical Processing, 2019: 199−208.
38	YUE W C, CHEN X F, GUI W H, et al A knowledge reasoning fuzzy-Bayesian network for root cause analysis of abnormal aluminum electrolysis cell condition. Frontiers of Chemical Science and Engineering, 2017, 11 (3): 414- 428. doi: 10.1007/s11705-017-1663-x
39	LEI Y X, CHEN X, MIN M, et al A semi-supervised Laplacian extreme learning machine and feature fusion with CNN for industrial superheat identification. Neurocomputing, 2019, 381, 186- 195.
40	CAO D Y, ZENG S P, LI J H Research on prediction model of aluminum electrolyte superheat degree. Light Metals, 2010, 50 (10): 35- 38.
41	HAO X C, WANG Z X, SHAN Z Y, et al Prediction of electricity consumption in cement production: a time-varying delay deep belief network prediction method. Neural Computing and Applications, 2019, 31, 7165- 7179. doi: 10.1007/s00521-018-3540-z
42	SHAMSOLMOALI P, KUMAR JAIN D, ZAREAPOOR M, et al High-dimensional multimedia classification using deep CNN and extended residual units. Multimedia Tools and Applications, 2019, 78 (17): 23867- 23882. doi: 10.1007/s11042-018-6146-7
43	LI Z, LI J Y, WANG Y, et al A deep learning approach for anomaly detection based on SAE and LSTM in mechanical equipment. The International Journal of Advanced Manufacturing Technology, 2019, 103, 499- 510.
44	LIU Y S, XIA S Y, YU H, et al Prediction of aluminum electrolysis superheat based on improved relative density noise filter SMO. Proc. of the IEEE International Conference on Big Knowledge, 2018, 376- 381. doi: 10.1109/ICBK.2018.00057
45	LEI Y X, CEN L H, CHEN X F, et al A hybrid regularization semi-supervised extreme learning machine method and its application. IEEE Access, 2019, 7, 30102- 30111. doi: 10.1109/ACCESS.2019.2900267
46	YANG Y, WU Q M J Multilayer extreme learning machine with subnetwork nodes for representation learning. IEEE Trans. on Cybernetics, 2016, 46 (11): 2570- 2583. doi: 10.1109/TCYB.2015.2481713

Dataset	Training data	Testing data	Attribute	Class
Iris	90	60	4	3
Glass	128	86	10	6
Ecoli	217	119	7	8
Breast-can	455	114	30	2
Balance	500	128	4	3
Diabetes	614	154	8	2
Proteins	1 186	296	56	10
Waveform	2 500	2 500	21	3

Dataset	ELM	C-ELM	V-ELM	CV-ELM
Iris	93.3	96.0	94.7	97.1
Glass	91.6	92.5	94.4	95.3
Ecoli	85.7	86.3	88.1	89.3
Breast-can	94.1	95.9	96.7	98.4
Balance	90.0	93.4	91.4	95.5
Diabetes	76.2	79.0	78.5	82.9
Proteins	87.3	89.9	90.6	92.3
Waveform	85.4	86.0	86.9	87.4

Dataset	ELM	C-ELM	V-ELM	CV-ELM
Iris	0.009 8	0.015 6	0.078 1	0.093 8
Glass	0.312 5	0.046 8	0.184 3	0.246 8
Ecoli	0.040 9	0.048 5	0.193 7	0.303 1
Breast-can	0.063 1	0.089 7	0.504 9	0.533 4
Balance	0.085 9	0.091 7	0.386 5	0.465 3
Diabetes	0.094 5	0.137 4	0.780 9	0.908 3
Proteins	0.145 9	0.185 6	1.617 1	1.962 8
Waveform	0.352 8	0.594 0	2.859 4	3.067 5

Dataset	SVM		BP		Deep LSTM		CV-ELM
Dataset	Accuracy/ %	Training time/s	Accuracy/ %	Training time/s	Accuracy/ %	Training time/s	Accuracy/ %	Training time/s
Iris	93.2	9.5931	93.0	2.860 5	96.4	10.987 5	97.4	0.093 8
Glass	94.0	20.027 5	93.2	6.517 5	94.1	18.913 5	95.3	0.246 8
Ecoli	85.9	13.576 8	80.4	4.971 9	83.6	11.123 1	89.3	0.303 1
Breast-can	95.3	19.003 7	96.2	9.154 1	96.7	17.642 2	98.4	0.533 4
Balance	95.7	16.315 9	91.8	5.847 8	92.3	19.322 5	95.5	0.465 3
Diabetes	77.3	28.219 7	74.9	12.517 5	80.2	29.284 4	82.9	0.908 3
Proteins	90.7	57.015 3	87.1	19.938 1	93.5	30.987 5	92.3	1.962 8
Waveform	86.2	69.476 6	86.4	14.251 6	85.8	52.357 5	87.4	3.067 5

Number	Variable	Description
1	${x_1}$	Current working voltage
2	${x_2}$	Bath resistance
3	${x_3}$	Content of Iron
4	${x_6}$	Content of Silica
5	${x_5}$	Electrolyte level
6	${x_6}$	Molecular ratio
7	${x_7}$	Average bath voltage
8	${x_8}$	Addition amount of AlF₃

Constrained voting extreme learning machine and its application

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

Share this article

Figures/Tables 17

References 46

Related Articles 2

Recommended Articles

Metrics

Comments

Evaluation index	AE	DBN	SAE	CNN	RF	ELM	ML-ELM	CV-ELM
RMSE	0.546	0.303	0.415	0.447	0.318	0.322	0.32	0.259
Accuracy	0.692	0.83	0.765	0.748	0.823	0.806	0.819	0.874
Training time	—	9.674	7.257	21.843	13.487	0.172	1.34	0.857

[1]	Xiaokai Liu, Rong Li, Chenglin Zhao, and Pengbiao Wang. Robust signal recognition algorithm based on machine learning in heterogeneous networks [J]. Journal of Systems Engineering and Electronics, 2016, 27(2): 333-342.
[2]	Yongping Zhao and Kangkang Wang. Fast cross validation for regularized extreme learning machine [J]. Journal of Systems Engineering and Electronics, 2014, 25(5): 895-900.