Overtraining in fuzzy ARTMAP: myth or reality?

Abstract

We examine the issue of overtraining in fuzzy ARTMAP. Over-training in fuzzy ARTMAP manifests itself in two different ways: 1) it degrades the generalization performance of fuzzy ARTMAP as training progresses; and 2) it creates unnecessarily large fuzzy ARTMAP neural network architectures. In this work we demonstrate that overtraining happens in fuzzy ARTMAP and propose an old remedy for its cure: cross-validation. In our experiments we compare the performance of fuzzy ARTMAP that is trained: 1) until the completion of training, 2) for one epoch, and 3) until its performance on a validation set is maximized. The experiments were performed on artificial and real databases. The conclusion derived from these experiments is that cross-validation is a useful procedure in fuzzy ARTMAP, because it produces smaller fuzzy ARTMAP architectures with improved generalization performance. The trade-off is that cross-validation introduces additional computational complexity in the training phase of fuzzy ARTMAP