69.147.112.169
On September 11, 2001, the air was sizzling with anger — and the anger got hotter as the hours passed. That, anyway, was one finding of a 2010 analysis by Mitja Back, Albrecht Küfner, and Boris Egloff of 85,000 pager messages sent that day. The researchers employed a commonly used tool called Linguistic Inquiry and Word Count, or LIWC, which teases out information from the frequency of word usages in texts.
But were Americans really so angry? Clemson University psychologist Cynthia L. S. Pury wasn’t out to answer that question when she made the discovery that was just published online in Psychological Science, a journal of the Association for Psychological Science.
A researcher on courage, Pury thought the data set would be “ideal” to unearth signs of that virtue during those terrible hours — either of people springing to action or praising others who did. “I downloaded the whole thing and started playing around with it in Excel,” she says. But because that program shows the messages line by line, instead of aggregating it as LIWC does, Pury quickly saw that Back and his colleagues had drawn inaccurate conclusions: “The apparent rise in anger was due to a single message repeated thousands of times. It contained the word ‘critical,’ which LIWC read as indicating anger. However, in this case it meant ‘urgent’ or ‘very important.’ In fact, the message did not even seem to be from a person, but rather from a computer system indicating that critical maintenance was needed.”
Eliminating those repetitions, the dramatic rise in anger disappeared over time.
Comments Pury, “LIWC can be quite sophisticated in looking for words,” but without a way to read and analyze content for meaning, it can lead users far astray.
The experience compelled Pury to publish her findings in a letter warning other researchers to be wary of LIWC and similar tools “before they find a way of cleaning the data” of redundancies and other glitches.
While the discovery put the kibosh on one line of inquiry for the time being, it also made Pury feel better about her fellow Americans. “The anger findings made sense to me when I read them,” she says, “but thinking back to 2001, it was not how the situation seemed at the time.” All around Pury, “people were sad and scared. But they were also incredibly considerate and warm and appreciative of each other.” Indeed, the data revealed those feelings. Among the most frequent messages that showed up in Excel was one expressing a fierce need to connect: “Call me.”
For more information about this study, please contact: Cynthia L. S. Pury, Clemson University, at [email protected].
The APS journal Psychological Science is the highest ranked empirical journal in psychology. For a copy of the article “Automation Can Lead to Confounds in Text Analysis: Back, Küfner, and Egloff (2010) and the Not-So-Angry Americans” and access to other Psychological Science research findings, please contact Divya Menon at 202-293-9300 or [email protected].
