TY - GEN
T1 - A new approach to improving generalization ability of feed-foward neural networks
AU - HUA, Qiang
AU - GAO, Yue
AU - WANG, Xi-Zhao
AU - ZHAO, Bo-Yi
N1 - The comments and suggestions from the reviewers greatly improve this paper. This work is supported by the national natural science foundation of China (60903088, 60903089,), by the natural science foundation of Hebei Province (F2008000635, F2009000227), by the key project foundation of applied fundamental research of Hebei Province (08963522D).
PY - 2010
Y1 - 2010
N2 - In order to improve the generalization ability of feed-forward neural networks, a new objective function of learning procedure for training single hidden layer network is proposed. This objective function is composed of two information entropy, one is the cross entropy as the main optimization term and the other is the fuzzy entropy as the regularization term. In this paper, we are fused the concept of entropy to the network training process by the regularization method. We also derive the new learning rule of neural network. Our experimental results show that the generalization ability of networks by the proposed algorithm is better than other well-known learning methods in the same time complexity.
AB - In order to improve the generalization ability of feed-forward neural networks, a new objective function of learning procedure for training single hidden layer network is proposed. This objective function is composed of two information entropy, one is the cross entropy as the main optimization term and the other is the fuzzy entropy as the regularization term. In this paper, we are fused the concept of entropy to the network training process by the regularization method. We also derive the new learning rule of neural network. Our experimental results show that the generalization ability of networks by the proposed algorithm is better than other well-known learning methods in the same time complexity.
KW - Feed-forward neural networks
KW - Generalization ability
KW - Gradient descent method
KW - Information entropy
KW - Regularization
UR - http://www.scopus.com/inward/record.url?scp=78149306610&partnerID=8YFLogxK
U2 - 10.1109/ICMLC.2010.5580852
DO - 10.1109/ICMLC.2010.5580852
M3 - Conference paper (refereed)
AN - SCOPUS:78149306610
SN - 9781424465279
T3 - International Conference on Machine Learning and Cybernetics (ICMLC)
SP - 1413
EP - 1419
BT - Proceedings : 2010 International Conference on Machine Learning and Cybernetics, ICMLC 2010
PB - IEEE
T2 - 2010 International Conference on Machine Learning and Cybernetics, ICMLC 2010
Y2 - 11 July 2010 through 14 July 2010
ER -