So far, film studios have had to put in huge amounts of effort to set monsters, superheroes, fairies or other virtual characters into real feature film scenes. Within the so-called motion capturing process, real actors wear skintight suits with markers on them. These suits reflect infrared light that is emitted and captured by special cameras. Subsequent to this, the movements of the actors are rendered with the aid of software into animated characters. The most popular example of this is “Gollum” from the film Lord of the Rings, played by Andy Serkis.
Nils Hasler and Carsten Stoll, two researchers from the Max Planck Institute for Informatics in Saarbruecken, developed a method that works without markers. It immediately transfers actors’ movements to the virtual characters in near real-time. They presented their technology for the first time at the Cebit in 2013. The researchers used the last few months to improve their technology. “We made it more user-friendly”, explains Nils Hasler. “Now it is possible to film the movie scenes outdoors and not only in the studio.” For example, movie scenes like those from Lord of the Rings, where Gollum has to wander through the mountainous landscape of Middle Earth, wouldn’t need to be produced in the studio any longer, but could actually get filmed in the landscape where the scene takes place.
Moreover, the researchers dealt with the task of transferring the movements of two actors at the same time into two animated characters. “But the software needs a little bit more computing time to deal with two persons”, explains Carsten Stoll. Moreover, the technique makes it possible to imitate entire tracking shots. The movements of one character can thus be more easily captured from every angle.
Together with Christian Theobalt, leader of the research group “Graphics, Vision & Video” at the Max Planck Institute for Informatics, they could solve another problem: with their method, they can display people in full even if they had been partly obscured by other characters in a movie scene. This novel approach is not only interesting for the film and gaming industry. Sports journalists could better comment on movement sequences during a live boxing match or judo coaches could use it to analyze the fighting techniques of their athletes immediately. “Company doctors and physiotherapists could also use the technique to prevent, for example, back pain issues for company staff, or to optimize work processes”.
To commercialize their technology, Hasler and colleagues founded the company “The Captury” in last June. At the same, they are already working on first requests from industry. The researchers also won the main award within the IKT founder competition that is announced yearly by the Federal Ministry of Economy and Energy and includes the prize of 30,000 Euro.