Adaptive Batch Variance-Reduced Optimization for Online Self-Organizing RBFNNs with Incremental Learning

This paper proposed an adaptive batch variance-reduced optimization method for training online self-organizing radial basis function neural networks (RBFNNs) with incremental learning capabilities. Traditional RBFNNs often face challenges in balancing computational efficiency and gradient estimation accuracy, particularly in dynamic environments where data arrives sequentially. The proposed method addresses this by dynamically adjusting the batch size based on the variance of gradient estimates, thereby reducing noise while maintaining training efficiency. The RBFNN architecture is designed to grow or adapt its neurons in response to incoming data, ensuring flexibility without retraining the entire model. A key innovation lies in integrating variance-reduced gradient techniques, such as stochastic variance reduced gradient (SVRG), with online self-organizing mechanisms to achieve stable and fast convergence. Furthermore, the incremental learning framework enables the model to continuously update its parameters using new data, eliminating the need for full dataset retraining. Experimental validation demonstrates that our approach significantly improves both training stability and prediction accuracy compared to conventional methods. The method is particularly suitable for real-time applications where data distributions evolve over time, offering a scalable and robust solution for adaptive learning systems. This work contributes to the broader field of online machine learning by providing a theoretically grounded yet practical framework for training self-organizing neural networks under non-stationary conditions.