Crowdsourcing annotation: modelling keywords using low level features

Abstract

Tagging large collections is often prohibitive and manual tags are known to be imprecise, ambiguous, inconsistent and subject to many variations. A possible way to alleviate these problems and improve the annotation quality is to obtain multiple annotations per image by assigning several annotators into the task. In the current work we present an approach to model the view of several annotators using four MPEG-7 descriptors and a well known data classifier. We apply keywords modelling to the annotation data collected in the framework of Commandaria project where sixteen non-expert users annotated a set of a hundred images using a predefined set of keywords. The images sharing a common keyword are grouped together and used for the creation of the visual model corresponds to this keyword. Finally, the created models used to classify the images into the keyword classes in terms of 2-classes combinations using the 10-fold cross-validation technique. The experimental results are examined under two perspectives: First, in terms of the separation ability of the various keyword classes and second, in terms of the efficiency of the four visual descriptors as far as the image classification task is concerned