After nearly two years of a global pandemic, with endless days spent working remotely, we’re all craving physical contact with others.
For good reason, too: As human beings, touch is one of the most important social and emotional cues we have. It can silently communicate volumes – a reassuring pat on the back; a sharp poke of warning; an empathetic squeeze of the arm. But in the age of social distancing, how do we maintain those sorts of subtle connections?
“That’s something everyone has been very aware of lately,” says Allison Okamura, professor of mechanical engineering at Stanford. “For a long time, you couldn’t shake hands, you couldn’t hug a loved one. That touch is an important part of mental health. It creates a sense of closeness with other people.”
Okamura studies haptics, a field devoted to creating touch feedback between humans and machines. Today, she says, there are plenty of devices that can send a “touch” over the airwaves or the internet on a very rudimentary level: Your smartphone, for instance, can buzz in different patterns and intensities, each of which can convey different meanings. When it comes to conveying emotion, however, that tiny buzzer falls flat.
Remotely transmitting all the subtle nuances of social touch might eventually be possible, but there are a few unique challenges along the way. First, it requires figuring out what a social touch really is in the first place, and deciphering how someone physically interacts with a partner to convey happiness, fear, kindness and so on. Then, researchers would have to build a small wearable device that can deliver an approximate version of those sensations. Finally, it requires translating a vast amount of information about a touch gesture – its exact pressures, movements and undulations – into signals that move just a few actuators in that device. It’s not exactly easy.
In a new study published in the journal IEEE Transactions on Haptics, Okamura’s team took a step closer to that goal. After testing on a few dozen volunteers, they developed a basic proof of concept for a “social haptics” device: a homemade sleeve with eight small actuators sewn inside. By manipulating those actuators in distinct patterns, the team was able to give the sleeve’s wearer sensations that they could often identify as having specific emotional content.
Okamura is quick to note that the device doesn’t mimic social touch precisely. As long as the person on the receiving end understands the intent of the “touch” it provides, though, it might be enough. “It just needs to create a haptic illusion,” she says. “It might feel like someone brushing your arm, but it really comes from a few distinct actuators moving in a particular way.”
Introducing “Haptic Emojis”
As part of the study, Okamura’s team recruited pairs of volunteer subjects, either romantic couples or close friends, who would be comfortable and familiar with being touched by a partner. The group brought them two at a time into a small conference room decked out like an intimate coffee shop: Round wooden stools sat on a shag carpet between potted trees, bathed in soft mood lighting, while a folding wooden screen formed a cozy backdrop. The team fitted one of each pair with a specialized sleeve of pressure sensors, then asked their partner to perform dozens of social touch gestures on them.
“We gave them specific cues or prompts – not just ‘touch their arm like you’re happy for them,’ but entire scenarios that helped put them into the right mindset,” says Mike Salvato, a PhD student in Okamura’s lab and first author on the paper. “There were extended stories about things like, ‘you had a bad day at work and came home to an understanding partner,’ or ‘you felt grateful that a friend intervened during an awkward conversation,’” thus sparing them from committing a social faux pas, Salvato said.
Over and over, Salvato used a complex algorithm to track where subjects were touching the sensors. The algorithm then used the data to “compress” each gesture, breaking it down into signals that could be sent through the team’s custom-built actuator sleeve. Inside, eight voice coils (the magnetic part of a speaker that moves in response to electrical signals) flanked the wearer’s arm, ready to squeeze, shake or tap. Using this device, Salvato and team then played back the distilled touch information, and waited to see if the person wearing the sleeve could recognize its emotional intent. “It actually worked a lot better than I thought it would,” Salvato notes.
Salvato calls this sort of truncated shorthand touch a “haptic emoji.” Just like its pictograph counterparts, it doesn’t convey a full message, but it does provide just enough information to know what its sender intended.
“Think of it this way,” Salvato said. “If you get a sad face emoji and you know your partner is at a job interview, you can already get a picture of how things went. Likewise, if I send my partner a cat emoji, they know what it means even though other people definitely wouldn’t. It’s like a social shorthand between us. I think these haptic emojis can do something similar.”
Bringing People Closer Together
At the moment, this work is still in its very early stages. The translation between real-world touch and its stripped-down robotic counterpart isn’t happening in real time, and only prerecorded “touch emojis” can be sent to a recipient wearing a haptic sleeve. In the future, however, that could change.
“It would be more like direct messaging rather than emoji, where someone would be able to touch a device with sensors, and then immediately transfer that information over to a device on the other end that could play it back to someone remotely,” says Heather Culbertson, who was a postdoctoral researcher in Okamura’s lab when the study was originally conceived. (Culbertson is now an assistant professor of computer science at the University of Southern California.) “I could see it augmenting our interactions when being face-to-face isn’t a possibility. It might be used extensively for elderly people in nursing homes – there’s plenty of research showing that touch is really important for their mental and physical health,” she says.
When most people hear about the possibility of two-way touch communication, Okamura adds, the first thought that usually comes up is a fully virtual world, where a sense of touch is recreated completely. Although the idea looms large in the popular imagination (think Ready Player One), she’s adamant that the future of haptic technology isn’t just creating lifelike virtual universes.
“Our intent is here in the real world. This work isn’t meant to replace touch, but enhance it,” she says. “It stands in where real touch can’t, the way a video call or virtual meeting can help recreate an in-person experience if we’re separated from our loved ones by COVID-19, or by thousands of miles. Ultimately, we want to create devices that that help people communicate and bring them closer together.”
