Accounting for diversity in subjective judgments

Abstract

In this paper we argue against averaging as a common practice in the analysis of subjective attribute judgments, both across and within subjects. Previous work has raised awareness of the diversity between individuals' perceptions. In this paper it will furthermore become apparent that such diversity can also exist within a single individual, in the sense that different attribute judgments from a subject may reveal different, complementary, views. A Multi- Dimensional Scaling approach that accounts for the diverse views on a set of stimuli is proposed and its added value is illustrated using published data. We will illustrate that the averaging analysis provides insight to only l/6th of the total number of attributes in the example dataset. The proposed approach accounts for more than double the information obtained from the average model, and provides richer and semantically diverse views on the set of stimuli.