A hierarchical classification scheme for semantic image annotation

Abstract

In this paper we address some of the issues commonly encountered in automatic image annotation systems such as simultaneous labeling with keywords corresponding to both abstract terms and object classes, multiple keyword assignment, and low accuracy of labeling due to concurrent categorization to multiple classes. We propose a hierarchical classification scheme which is based on predefined XML-dictionaries of tree form. Every node of such a tree defines a particular classification task while the children of the node correspond to classification categories. The winning class (subnode) defines the subsequent classification task and the process continues until the leafs of the tree are reached. The final classification task is performed at image segment level; that is every image segment is assigned a particular keyword corresponding to a tree leaf. The path followed from the root of the XML tree to the leafs along with the union of labels assigned to the image segments compose the list of annotation keywords for the input image. The performance of the proposed method was tested on a set of 1046 images, taken from the athletics domain, containing a total of 3546 concept instances of 33 different concepts. The results promising and show the potential of the divide and conquer approach we follow through the proposed hierarchical classification scheme