Please use this identifier to cite or link to this item:
Title: Validating Wordscores: The Promises and Pitfalls of Computational Text Scaling
Authors: Bruinsma, Bastiaan 
Gemenis, Kostas 
Major Field of Science: Social Sciences
Field Category: Media and Communications
Keywords: Wordscores;Text Scaling
Issue Date: 3-Jul-2019
Source: Communication Methods and Measures, 2019, vol. 13, iss. 3, pp. 212 - 227
Volume: 13
Issue: 3
Start page: 212
End page: 227
Journal: Communication Methods and Measures 
Abstract: Wordscores is a popular computational text analysis method with numerous applications in communication research. Wordscores claims to scale documents on specified dimensions without requiring researchers to read or even understand the language of the input text. We investigate whether Wordscores delivers this claim by scaling the Euromanifestos of 117 political parties across 23 countries on 4 salient dimensions of political conflict. We assess validity by comparing the Wordscores estimates to expert surveys and other judgmental measures, and by examining the Wordscores’s estimates ability to predict party membership in the European Parliament groups. We find that the Wordscores estimates correlate poorly with expert and judgmental measures of party positions, while the latter outperform Wordscores in the predictive validity test. We conclude that Wordscores does not live up to its original claim of a “quick and easy” language blind method, and urge researchers to demonstrate the validity of the method in their domain of interest before any empirical analysis.
ISSN: 19312458
DOI: 10.1080/19312458.2019.1594741
Rights: © The Author(s)
Attribution-NonCommercial-NoDerivatives 4.0 International
Type: Article
Affiliation : Scuola Normale Superiore 
Max Planck Institute for the Study of Societies 
Appears in Collections:Άρθρα/Articles

Files in This Item:
CORE Recommender
Show full item record

Citations 50

checked on Nov 6, 2023


Last Week
Last month
checked on Oct 29, 2023

Page view(s) 50

Last Week
Last month
checked on Mar 2, 2024

Download(s) 50

checked on Mar 2, 2024

Google ScholarTM



This item is licensed under a Creative Commons License Creative Commons