Improving the Performance of Resource Allocation Networks through Hierarchical Clustering of High-Dimensional Data

Abstract

Adaptivity to non-stationary contexts is a very important property for intelligent systems in general, as well as to a variety of applications of knowledge based systems in era of “ambient intelligence”. In this paper we present a modified Resource Allocating Network architecture that allows for online adaptation and knowledge modelling through its adaptive structure. As in any neural network system proper parameter initialization reduces training time and effort. However, in RAN architectures, proper parameter initialization also leads to compact modelling (less hidden nodes) of the process under examination, and consequently to better generalization. In the cases of high-dimensional data parameter initialization is both difficult and time consuming. In the proposed scheme a high – dimensional, unsupervised clustering method is used to properly initialize the RAN architecture. Clusters correspond to the initial nodes of RAN, while output layer weights are also extracted from the clustering procedure. The efficiency of the proposed method has been tested on several classes of publicly available data (iris, ionosphere, etc.)