Sword fights are often the weak link in virtual reality (VR) fighting games, with digital avatars engaging in battle using imprecise, pre-recorded movements that barely reflect the player’s actions or intentions. Now a team at the University of Bath, in collaboration with the game development studio Ninja Theory, has found a solution to the challenges of creating realistic VR sword fights: Touché – a data-driven computer model based on machine learning.Dr Christof Lutteroth, who created Touché with colleague Dr Julian Padget and EngD student Javier Dehesa, said: “Touché increases the realism of a sword fight by generating responsive animations against attacks and eliminating non-reactive behaviour from characters.

“Using our model, a game character can anticipate all possible fight situations and react to them, resulting in a more enjoyable and immersive game experience.”

The unpredictability of user actions presents a major conundrum for designers of VR games, explained Dr Lutteroth, who is a senior lecturer in Computer Science, director of Real and Virtual Environments Augmentation Labs (REVEAL) and co-investigator at the Centre for the Analysis of Motion, Entertainment Research and Applications (CAMERA). “VR games offer new freedom for players to interact naturally using motion, but this makes it harder to design games that react to player motions convincingly,” he said.

He added: “There are different expectations for screen-based video games. With these, a player presses ‘attack’ and their character displays a sequence of animations. But in a VR game, the player input is much harder to process.”

The Touché framework for VR sword fighting simplifies the necessary technical work to achieve a convincing simulation. It eliminates the need for game designers to add layer upon layer of detail when programming how a character should move in a particular situation (for instance, to block a particular sword attack). Instead, actors wearing motion capture equipment are asked to perform a range of sword fighting movements, and Touché builds a model from these movements. The virtual version of the actor is able to react to different situations in a similar fashion to a flesh-and-blood fighter. Game designers can then fine-tune this model to meet their needs by adjusting high-level parameters, such as how skilled and aggressive the game character should be. All this saves game developers a lot of time and leads to more realistic results.

For the Bath study, 12 volunteers were asked to take part in two three-minute sword fights: for the first fight, they used technology that is currently available and for the second, they used Touché. Touché had a strong positive effect on realism and the perceived sword fighting skills of game characters. Feedback from participants pointed to a convincing preference for Touché, with current sword fights being described as ‘unresponsive’ and ‘clumsy’ by comparison.

“Based on this, we are convinced that Touché can deliver more enjoyable, realistic and immersive sword-fighting experiences, presenting a more skilled and less repetitive opponent behaviour,” said Dr Lutteroth. “I’m convinced this framework is the future for games – not only for sword fighting but also for other types of interaction between game characters. It will save developers a lot of time.”

Javier Dehesa Javier, who is based at the Centre for Digital Entertainment, interviewed game developers who had tested this new technology. He said: “Developers see the Touché framework as an important practical step in the industry towards data-driven interaction techniques. We could see this technology appear in commercial games very soon.”

Touché: Data-Driven Interactive Sword Fighting in Virtual Reality is published by CHI ’20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems.