Imagine strapping on a bit of virtual-reality gear and becoming a T-rex – not just looking like one, but feeling like it from the inside out. That could be pretty cool, right?
After a few minutes of practice you become comfortable in your new skin. Your arms and hands grow stubby, with little of the range and dexterity you’re used to. But elsewhere, your new body is full of compensations. Your dainty human mouth is replaced by a fearsome set of jaws that you can open almost 90 degrees. Your legs are huge, long and powerful with a wide, splayed stance. Your spine is now stiff and sinewy, and you can pivot your whole body forward and backward about your hips like a seesaw. And last but not least, a huge, heavy tail that you can control with amazing grace and speed swooshes around behind you, giving you a whole new sense of balance and maneuverability.
Your sense of being a T-rex is so total, so absorbing, that you may completely lose touch with the two-armed, two-legged, bipedal modified-primate body you grew up with. This VR simulation hasn’t simply superimposed a saurian-shaped avatar in the location of your body; it has actually reconfigured your felt body sense. Your intuitive workaday understanding of your real-life body’s balance, placement and proportion – what neuroscientists call your body schema – has faded to the periphery of your awareness and been replaced by a saurian one.
When people imagine what a mature VR implementation will look like, they mostly imagine three-dimensionalized twists on present-day applications for gaming, education, social networking, business development, porn, and so on. And it’s true. All those things will come to pass. World of Warcraft, Second Life, MySpace, and every other big online phenomenon will have some sort of analog-on-steroids that capitalizes on VR’s ability to bring a person’s whole sense of embodiment into any imaginable environment.
What’s more, the delight taken by many online role-playing-game players in choosing avatars of different gender, height, race, appearance, even species will reach new heights. In VR you won’t merely command your avatar about on a small flat panel display while you look down from on high; you will “wear” it like a costume. In cyberspace you can be male, female, short, tall, sexy, plain, realistic, cartoonish. You can be an elven knight, a Confederate soldier, an anime cat-girl, a robot, or anything else you could wish to appear as.
But even that is just a starting point. Treating avatars like costumes is still playing by the rules of your native body schema. But what if you could slip into an avatar with a radically different schema? Could you become a horse? An eagle? An octopus? A T-rex? Some of these may prove more difficult to achieve than others, but it has become clear that new body schemas can be recreated and experienced thanks to the brain’s willingness to surrender its familiar body schema in response to novel patterns of sensory input.
To understand how this transformation will occur, you need to appreciate the sprawling network of body maps that are etched into your brain’s thinly layered surface.
The largest and most basic of these is your primary touch map. It has swaths of tissue dedicated to mapping touch sensations from each finger, hand, cheek, leg, arm, foot and toe, as well as your tongue, teeth, throat, genitals, and every other body part you can name. When someone claps you on the shoulder, you know it is your shoulder and not your neck or your arm because the cells that make up your shoulder map become active.
You also have a motor map (“motor” being a fancy term for movement). It has swaths of tissue dedicated to the various muscle groups in your fingers, hands, arms, legs, neck and so on. You can choose which finger to wiggle because each finger is represented separately in your primary motor map. The cells in the chosen finger map fire, sending commands down to your muscles to make the intended movement happen.
Your body schema is your brain’s online, real-time model of your body based on sensory information gathered from your skin, bones, joints, tendons and muscles. It includes the overall layout, position and speed of your limbs, head and other body parts; your reach, strength, stature, weight, girth and balance; what your eyes tell you about your body’s position and appearance; your body’s vast, unconscious library of “muscle memories”; and your ability at any given time to move around and interact with your environment.
You also map the space around your body. Wave your arm up over your head, out to your side and down to your leg. Each point of that space is mapped inside your brain in relation to your body’s physical capabilities. Such higher body maps are also what allow you form and execute goals, plans and intentions. In concert, these maps generate your body schema.
Most of the time, your body map network is tightly integrated. Your body schema feels constant moment to moment because your corporeal body is pretty much immutable. Your daily experience of sights, sounds and physical sensations follows a certain set of patterns that you consider normal, a given, even “real.”
And yet your body schema is remarkably flexible. If you change the patterns of sensory input and sensory-motor feedback your brain receives, your mind will automatically do its best to come up with a new interpretation of your body that best “explains” those novel patterns – even if it contradicts the old schema.
You can gain an appreciation of your body schema’s fundamental slipperiness through somatic illusions. Using simple tools like buzzers, mirrors and blindfolds you can reconfigure your body sense. You can make your nose, limbs or waist feel like they are stretching or shrinking. You can convince your body schema that your joints are bending impossibly backward, or passing through walls. By fine-tuning the synchronization between two buzzers in each hand, you can create the illusion that you are sensing the vibrations in the empty space between your hands. With another simple setup, you can experience touch sensations applied to a fake rubber hand lying on a table across the room from you as vividly as if it were actually part of your body.
If you can fool your body schema that easily with simple props and parlor tricks, imagine what will be possible with a mature VR implementation, with its complete control of visual and auditory sensation. Add computer-controlled tactile stimulation and force-feedback into the mix, and the possibilities are vast.
For example, how would you like to be a lobster? Jaron Lanier, the original coiner of the term virtual reality and one of the field’s foremost pioneers, smiles as he thinks back to the early 1980’s and recalls what it was like to have lobster legs sprouting from his body.
As he had done hundreds of times before, Lanier snugged the virtual headset over his eyes and found himself standing in a virtual room. But this time his avatar was not entirely human. He held out his arms, and as usual, his avatar’s arms did the same thing. But looking down at his rib cage he could see six segmented legs, twitching gently like reeds being tousled by a breeze. He set about figuring out how to control them.
As Lanier experimentally moved different parts of his body (his real body), the lobster legs bent and waved in ways that seemed random at first. But in fact there was a formula to it. The computer that was rendering his avatar was also tracking the bending of his joints (his real joints) and feeding that information into a composite control signal for the lobster legs. The recipe was complex. A subtle combination of the angles of the left wrist, right knee, and right shoulder, for example, might contribute to the flexion of the “elbow” joint of the lower-right lobster leg. It was much too complex and subtle for his rational mind to grasp. But the inherent physical intelligence of his body map network was able to learn the patterns at a gestalt level and acquire the knack for controlling the new appendages. He soon mapped the lobster avatar into his own body schema.
The system did impose some moderate limitations on Lanier’s movements. So long as he wanted to retain control of the lobster arms, he had to move his real body according to certain patterns and postures that weren’t exactly the ones he would have chosen naturally. (An engineer would say that the lobster arms were robbing some of the “degrees of freedom” from Lanier’s movements.) But he was still able to navigate decently well and use his hands and arms to do other things. And in exchange, he was able to learn to control a new set of limbs.
“After a bit of practice I was able to move around and make the extra arms wag individually and make patterns of motion,” he says. “I was actually controlling them. It was a really interesting feeling.”
This was just one of many small informal experiments Lanier and his team ran in those long-ago days of Pac Man and Tron. They toyed with a variety of altered avatars to see how much distortion, reformation, augmentation, and general weirdness they could impose and still get the mind to accept the avatars as “my body.” They tried lengthened limbs, they tried shoulders at mid-torso, they tried giant hands and stretchy arms like the superhero Plastic Man. They were amazed at what the mind would accept.
In one experiment they created an avatar with a tentacle protruding from the midriff that the user could control using the lobster-arm technique, and used the phantom-floating-buzzer illusion mentioned earlier to make it feel as though the vibrations were being felt in the tip of the tentacle.
“When you get the visual and tactile experiences going together, it becomes just astonishingly convincing,” said Lanier. “When you combine somatic illusions with visual feedback like that, you just get to this whole new level. It’s like your homunculus is maximally stretched at that point.”
Lanier calls this phenomenon “homuncular flexibility,” in honor of the primary touch and motor homunculi, as a shorthand for the surprising adaptability the body schema shows.
To return to the become-a-T-rex simulation, the computer would just use a variation on the lobster-arm technique, borrow a few degrees of freedom from your body – especially your hands and arms, since your T-rex avatar has the manual dexterity of, well, a lizard – and repurpose them into a control signal for your saurian avatar. Your sense of being able to seesaw your body up and down could be achieved by exaggerating the vertical visual feedback you got for ducking down or up only slightly. Just as people stop noticing that their avatars’ hands are moving several times the distance as their real-life appendages, so would you come to believe that your head was dipping and rearing through several meters instead of just inches. And as for the swooshing-tail-balancing part of the illusion, a backpack filled with sliding computer-controlled weights should do the job handily.
How far can the homunculus be bent before it breaks? Quite far, apparently, though the question is just starting to get serious attention. There is a whole new world of embodiment out there to explore. When you are given the power to morph your body freely, what will you become?
Sandra Blakeslee and Matthew Blakeslee (a mother-son science writing team) are co-authoros of The Body Has a Mind of Its Own, published in September 2007 by Random House.
