If you find yourself shouting and gesticulating wildly if others can’t hear you during a Zoom call, you’re not alone. The more the video quality of an online meeting degrades, the louder we start talking, a new study by researchers at Radboud University and the Max Planck Institute for Psycholinguistics finds. People also tend to change up their gestures to compensate. Their findings were published today in the Royal Society Open Science journal.
When conversing over Zoom or Skype, we use some of the same tactics to make ourselves heard as we use in the real world, says James Trujillo, first author of the paper and a cognitive scientist at Radboud University and the Max Planck Institute for Psycholinguistics. ‘If you’re talking to someone in a busy area with a lot of background noise, you typically use gestures to support your speech, and you start talking louder. When we talk in a video call, any issues that make it harder for people to understand each other usually come from technical difficulties. However, we still tend to use similar methods to compensate for those issues.’
Analyzing video calls
For the study, Trujillo and colleagues set up video calls between twenty pairs of participants. The participants sat in separate rooms, and were told to have a casual conversation for 40 minutes. Over the course of the call, the quality of the video deteriorated in ten steps, from very clear to extremely blurry. The researchers tracked how participants altered their speech and gestured over the course of the call.
At first, when the video quality deteriorated, participants reduced their arm and body movements somewhat, researchers found. However, when the quality decreased further, movement started increasing. The rate, speed and size of the gestures increased quickly at first, and then reached a certain plateau. Those participants who used gestures during the video call would also increase by up to 5 decibels when the video quality started dropping. Even when the image quality decreased so much that participants could barely see each other, they continued to use gestures and talk at a louder volume.
Trujillo: ‘What this shows is that speech and gestures are integrated. People are compensating for losing out on visuals by adapting with louder speech and bigger gestures. Even when the image can barely be seen, people don’t suddenly stop gesturing. It’s similar to how people talk to each other on the phone: we don’t see each other, and during an involved conversation we still move and gesticulate.’
A fuller picture for future research
‘Previous research has shown that speech and gestures are linked, but ours is the first to look into how visuals impact our behavior in those fields,’ notes Trujillo. ‘It indicates that speech and gestures are dynamically adapted to our demands. Our findings suggest that speech and gesture are two channels of communication that are tightly linked.
‘What this study shows is that if you’re studying communicative behavior, you have to look at the full picture. Some researchers have argued that gestures are only an addition to speech, that they are not integral to it. If that were the case, we wouldn’t see people use louder speech to compensate for the loss of gestures, or the other way around. This study shows their connection works both ways. Further research into speech should not just look at factors like loudness and tone of voice, but also include these gestures to get a proper sense of people’s behavior.
