Imagine you are a police detective trying to identify a suspect wearing a trench coat, baggy pants and a baseball cap pulled low. Or imagine you are a fashion industry executive who wants to market virtual clothing that customers of all shapes and sizes can try online before they purchase. Perhaps you want to create the next generation of “Guitar Hero” in which the user, not some character, is pumping out the licks. The main obstacle to these and other pursuits is creating a realistic, 3-D body shape — especially when the figure is clothed or obscured.
“If you see a person wearing clothing, can the computer figure out what they look like underneath?” asked Michael Black, professor of the computer science at Brown.
Black and graduate student Alexandru Balan believe they have figured out how that can be done. The researchers have created a computer program that can accurately map the human body’s shape from digital images or video. This is an advance from current body scanning technology, which requires people to stand still without clothing in order to produce a 3-D model of the body.
With the new 3-D body-shape model, the scientists can determine a person’s gender and calculate an individual’s waist size, chest size, height, weight and other features.
Black and Balan debuted their findings this month at the European Conference on Computer Vision in Marseilles, France. Their paper is one of fewer than 5 percent of submitted manuscripts chosen for oral presentation at the prestigious international gathering.
A simulation of the new technology can be seen here (or click on this link: http://www.cs.brown.edu/~alb/scapeClothing/).
The potential applications are broad. Besides forensics and fashion, Black and Balan’s research could benefit the film industry. Currently, actors must wear tight-fitting suits covered with reflective markers to have their motion captured. The new approach could capture both the actors’ shape and motion, while doing away with the markers and suits.
In sports medicine, doctors would be able to use accurate, computerized models of athletes’ bodies to better identify susceptibility to injury. In the gaming world, it could mean the next generation of interactive technology. Instead of acting through a character, a camera could track the user, create a 3-D representation of that person’s body and insert the user into the video game.
Brown University has filed two provisional patents covering the research and its potential commercial applications.
The key insight for Black and Balan was when they learned they could divine clues about a person’s shape even with clothing. They created a computerized body model from 2,400 detailed laser range scans of men and women in minimal clothing. They found that by combining information from a person in multiple poses, the computer was able to infer the gender of the person and the 3-D body shape. They further refined the model by incorporating the computer’s detection of skin in the images.
“As I move, my clothes become loose or tight on different parts of my body,” Black said. “Each pose gives different constraints on the underlying body shape, so while a person’s body pose may change, his or her true shape remains the same. By analyzing the body in different poses, we can better guess that person’s true shape.”
The researchers stress the technique is not invasive; it does not use X-rays, nor does it actually see through clothing. The software makes an intelligent guess about the person’s exact body shape.
The Office of Naval Research, the Rhode Island Economic Development Corp. and the Intel Corp. funded the research.
http://news.brown.edu/pressreleases/2008/10/3-d
