The ability to quickly scan another person’s body language or expression to get a quick read on what they’re thinking or feeling is a handy trick that most humans possess. Show up late for dinner, and all it takes is a glance of your mother’s body language to know that you’re in trouble.
New research by a pair of Stanford scientists reveals that quantitative analysis of such nonverbal cues can indicate a person’s ability to learn and the strength of their creative skills.
The findings could improve the delivery of information through online learning, or help employers to assemble workers in teams that will produce the most innovative ideas.
The research, conducted by Jeremy Bailenson, an associate professor in the Communication Department, and Andrea Stevenson Won, a communication doctoral student, leveraged Stanford’s Virtual Human Interaction Lab (VHIL) to take the study of nonverbal behavioral cues into the modern age. For decades, scientists have observed human behavior as a window into the brain. These observations, however, were laborious, subject to bias and could possibly miss subtle, but important, patterns.
The researchers turned the study of nonverbal cues into a big data project by utilizing the VHIL’s common video-game cameras to measure the exact movements of participants’ bodies, limbs and heads. Working with a hundred subjects and recording at 30 frames per second, the experiment produced a mountain of data, which they parsed with a machine-learning model trained to objectively identify patterns that might sneak past the human eye.
They then applied the technique to an experiment that could reveal the role body language plays in how effectively one person can teach another. In the experiment, the results of which are published in the journal IEEE Transactions on Affective Computing, a “teacher” learned several principles of water efficiency – one aspect of the VHIL’s mission is to try to teach people to reduce their toll on the environment – and then had five minutes to teach the lesson to a “student,” while the camera recorded the exchange. The student then took an exam to show how much of the lesson he/she had absorbed.
The scientists repeated this scenario 50 times, and entered the camera data and test scores into their model to identify the behaviors that correlated with poor test scores.
“For our sample and our task, students with very extreme movements with their upper body tended to learn worse than others,” Bailenson said.
Improving your non-verbal behavior
The authors caution that this result provides nearly no insight into cause and effect. The student might be moving because he/she is bored. Or it could simply have been because they sneezed or had an itch, and they missed a key detail. Or they might have been sick.
“But the critical thing from a psychological perspective as well as an applications perspective is that regardless of whether we know the cause, we can detect whether people are about to learn or not,” Bailenson said. “This gives us the opportunity to devise ways to adjust in real time to improve learning.”
The model also showed that large, irregular movements of the teacher’s head and torso correlated to – and could predict – poor test scores.
“When I teach, I pace the entire time,” Bailenson said. “This data is showing that this is probably not a great strategy.”
The rhythm of creativity
The second study, published in the Journal of Nonverbal Behavior, was also conducted by observing people interact in the VHIL. This time, however, a couple jointly brainstormed ways to improve water conservation techniques. In this experiment, creativity was measured by the number of ideas that the duos produced.
Previous research has indicated that encouraging people to move synchronously improves team collaboration and test scores. Conducted by hand, these observational studies also lacked fine detail and were open to bias.
By measuring every angle of the participants’ bodies and limbs, Bailenson and Won discovered that the more synchronous the subjects’ head movements, the more creative ideas they churned out. Across the sample, gauging the synchronicity of head movements proved a powerful predictor of creative output.
“Finding evidence to link synchrony with better collaborative outcomes is particularly interesting,” Won said. “For our next study, we are aiming to both improve our measures – for example, detecting specific gestures – and better understand what is important about synchrony in two-person interactions.”
The authors think that refining this observation could allow employers to increase efficiency on projects that require creative thinking.
“Imagine trying to hire pairs to work on a project,” Bailenson said. “Give your dozen candidates a two-minute test to see which pairs synchronize and which ones don’t. So now you’ve got a basically innovative team detector.”
The next step in both areas is to see if making a person aware of his or her body movements can induce a behavioral change that will increase their score. The scientists are currently designing hardware – for instance, a sensor that would issue a “beep” when the person’s head and torso begin to move too much – and software to test the effects of intervention.
