Coding schemes as measurement instruments? an attempt to assess the psychometric properties of a coding scheme

Abstract

CSCL researchers rely heavily on coding schemes to analyze collaborative discourse. As Rourke and Anderson (2004) explain, examples of qualitative content analysis (QCA) research in which a coding scheme is developed methodically and validated systematically are rare. In this study, we attempted to conduct a validation of a coding scheme using established quantitative methods; this was an effort to foster the replicability and validity of a coding scheme designed to measure online collaborative learning. Exploratory Factor Analysis suggested a different factor structure than the one conceptualized in the coding scheme. The authors raise questions related to what are the most appropriate and acceptable procedures in establishing the validity of coding schemes used in QCA