Projects per year
This paper gives a formal convergence analysis on the mini-batch training algorithm for noise resilient radial basis function (RBF) networks. Unlike the conventional analysis which assumes that the mini-batch process is operated in a stochastic manner, we consider that the mini-batch training process is operated in a deterministic manner. The deterministic process divides the training samples into a number of fixed mini-batches, and the mini-batches are presented in a fixed order. This paper first states the noise resilient objective function for weight noise and weight fault. We then derive the mini-batch training algorithm for this noise resilient objective function. Our main contribution is the convergence analysis on the mini-batch training algorithm. We show that under the deterministic setting, the mini-batch training algorithm converges. The converged weight vector is asymptotically close to the optimal batch mode solution. Also, we derive the sufficient conditions (the learning rate range) for convergence. Our theoretical results can be applied to not only the noise resilient objective function but also a large class of objective functions.
|Journal||International Journal of Machine Learning and Cybernetics|
|Early online date||6 Apr 2022|
|Publication status||Published - Sept 2022|
Bibliographical noteThis work is partially supported by Key Project of Science and Technology Innovation 2030 supported by the Ministry of Science and Technology of China (Grant No. 2018AAA0101301), GRF-RGC General Research Fund CityU 11203820 (9042958) and CityU 11209819 (9042816).
- Convergence analysis
- Fault tolerance
- Mini-batch training
- Noise resilience
- Online training
- Radial basis function networks
- Weight fault
- Weight noise
FingerprintDive into the research topics of 'Convergence analysis on the deterministic mini-batch learning algorithm for noise resilient radial basis function networks'. Together they form a unique fingerprint.
- 1 Active
KWONG, S. T. W., KUO, C. J., WANG, S. & ZHANG, X.
1/01/21 → 30/06/24
Project: Grant Research