Methodological challenges in the analysis of voting advice application generated data

Abstract

Voting advice applications (VAA) have become an increasingly popular feature of electoral campaigns. VAAs are online tools that use survey techniques to measure the degree to which the policy preferences of citizens match those of political parties or candidates. In some cases, such as The Netherlands, VAA's can attract millions of respondents providing an incredibly rich source of mass public opinion data. As a result political scientists have begun to exploit such datasets and this is fuelling a burgeoning literature on the topic. To date, however, there has been surprisingly little research on the cleaning techniques used to filter out the many rogues entries that are known to be present in VAA generated datasets. This paper presents the various methods used for cleaning VAA generated datasets that have been used for empirical research. Two main techniques are used based on item response timers and pattern recognition techniques. We show why cleaning matters and the problems that flow from not establishing rigorous cleaning techniques. The problem as such is not exclusive to VAA data but is common to all web based research involving self-administered surveys. To that end the techniques we present could be generalisable beyond the specific case of VAA-generated datasets.