Journal of Systems Engineering and Electronics ›› 2006, Vol. 17 ›› Issue (2): 450-454.doi: 10.1016/S1004-4132(06)60076-5

• SOFTWARE ALGORITHM AND SIMULATION • Previous Articles     Next Articles

Novel Newton's learning algorithm of neural networks

Lorcg Ning & Zhang Fengli   

  1. Coll. of Electronic Engineering, Univ. of Electronic Science and Technology of China, Chengdu 610054, P. R. China
  • Online:2006-06-26 Published:2019-12-20

Abstract:

Newton's learning algorithm of NN is presented and realized . In theory, the convergence rate of learning algorithm of NN based on Newton's method must be faster than BP's and other learning algorithms, because the gradient method is linearly convergent while Newton's method has second order convergence rate. The fast computing algorithm of Hesse matrix of the cost function of NN is proposed and it is the theory basis of the improvement of Newton's learning algorithm. Simulation results show that the convergence rate of Newton's learning algorithm is high and apparently faster than the traditional BP method's, and the robustness of Newton's learning algorithm is also better than BP method's.

Key words: Newton's method, Hesse matrix, fast learning, BP method, neural network