Associations Measured = Stereotypes Conveyed? A Semantic Validation of Word Embedding-Based Measures of Implicit Group Stereotyping in Large Text Corpora

Word embedding-based measures are increasingly being used in computational communication research to assess how entities (such as individuals or social groups) are implicitly contextualized within mediated discourse. We argue that these corpus-level metrics, yet, lack a demonstration of their semantic validity and point out several challenges that preclude researchers from using the traditional ``gold-standard'' coding for its establishment. In this study, we propose and apply an alternative avenue, namely, to use the experimental survey logic to test a causal conjecture between human-perceived context and the implicit associations measured using word embeddings. We report the results of an application of this approach which uses texts and measures from a previous study investigating the implicit stigmatization of ethnic groups. Results indicate alignment between participants' perceived group contextualization and the respective estimations from a word embedding model across experimental conditions. We interpret this as evidence for semantic validity of word embedding-based measures of implicit stereotypical associations.