Yesterday, I ran across a fascinating 1993 paper by sociologist Kathleen Carley, Coding Choices for Textual Analysis: A Comparison of Content Analysis and Map Analysis.
Using the now antiquated term “map analysis” – what I would call semantic network analysis today – Carley explains:
An important class of methods that allows the research to address textual meaning is map analysis. Where content analysis typically focuses exclusively on concepts, map analysis focuses on concepts and the relationships between them and hence on the web of meaning contained within the text. While no term has yet to emerge as canonical, within this paper the term map analysis will be used to refer to a broad class of procedures in which the focus is on networks consisting of connected concepts rather than counts of concepts.
This idea is reminiscent of the work of Peter Levine and others (including myself) on moral mapping – representing an individual’s moral world view through a thoughtfully constructed network of ideas and values.
Of course, a range of methodological challenges are immediately raised in graphing a moral network – what do you include? What constitutes a link? Do links have strength or directionality? Trying to compare two or more people’s networks raises even more challenges.
While Carley is looking more broadly than moral networks, her work similarly aims to extract meaning, concepts, and connections from a text – and faces similar methodological challenges:
By taking a map-analytic approach, the researcher has chosen to focus on situated concepts. This choice increases the complexity of the coding and analysis process, and places the researcher in the position where a number of additional choices must be made regarding how to code the relationship between concepts.
On its face, these challenges seem like they may be insurmountable – could complex concepts such as morality ever be coded and analyzed in such a way as to be broadly interpretable while maintaining the depth of their meaning?
This conundrum is at the heart of the philosophical work of Ludwig Wittgenstein, and is far from being resolved philosophically or empirically.
Carley is hardly alone in not having a perfect resolution dilemma, but she does offer an interesting insight in contemplating it:
…by focusing on the structure of relationships between concepts, the attention of the researcher is directed towards thinking about “what am I really assuming in choosing this coding scheme?” Consequently, researchers may be more aware of the role that their assumptions are playing in the analysis and the extent to which they want to, and do, rely on social knowledge.
A network approach to these abstract concepts may indeed be inextricably biased – but, then again, all tools of measurement are. The benefit, then in undertaking the complex work of coding relationships as well as concepts, is that the researcher is more acutely aware of the bias.