Dim lighting, a jumble of conversations, and loud music: bar staff have to master numerous challenges when serving their customers.
In a crowded place, they have to identify who would like to place an order and who does not. A Bielefeld research team analysed how the body language of the potential customer helps the bartenders to achieve this. The team found that real-life observations were at odds with the widespread belief that customers wave for signalling that they would like to order a drink. Analysing the real-life data showed that it is crucial how customers position themselves at the bar counter. These findings were integrated in the ‘brain’ of the robotic bartender James. The Bielefeld scientists published their study in the online research journal ‘Frontiers in Psychology’.
The study is part of the EU project ‘James’ (Joint Action in Multimodal Embodied Systems). Professor Dr. Jan de Ruiter’s Psycholinguistics Research Group at Bielefeld University is one of the project partners. The project unites researchers from Bielefeld, Crete, Munich and Edinburgh in their efforts to develop a bartending robot. The robot is named ‘James’, after the research project. Despite its suggestive name, the robot James has no resemblance to a butler from Victorian times. Its head is a tablet computer showing big, comic-style eyes which can establish eye contact with the customers. Its mouth moves in sync with its speech. The one-armed metal body forming James’ torso is fixed behind the bar. James accepts drink orders, reaches for the drink using its arm and a four-fingered hand and serves the drinks to its customers.
The project aims at advancing technology such that James can display socially intelligent behaviour that humans take for granted in daily life. ‘In order to respond appropriately to its customers the robot must be able to recognise human social behaviour,’ explains Professor De Ruiter. James should be able to understand users who have no prior knowledge about the robot and who have not been briefed in any way. In the noisy environment of a night club, the system cannot rely on its language components only. Thus, the robot learns how to interpret body language. ‘Currently, we are working on the robot’s ability to recognise when a customer is bidding for its attention,’ says De Ruiter. ‘Thus, we have studied the process of ordering a drink in real life.’ The machine requires a clear-cut definition of which signals indicate an order and which do not. Without a proper definition, it will misinterpret its customers’ signals. In turn, it would annoy or discomfort people by responding inappropriately to their behaviour.
The researchers from Bielefeld video-recorded how customers managed to get the attention of a bartender for placing their orders. Recordings were made in pubs and clubs in Bielefeld (Germany), Herford (Germany) and Edinburgh (United Kingdom). The analysis of the recordings revealed which signals were commonly used and which were used rarely by the customers to attract the bartenders’ attention. Contrary to what people tend to think, only one in fifteen customers looked at their wallets to signal that they would like to place an order. And fewer than one in twenty-five customers gestured at the bartender. The most common and successful signals are less pronounced: more than ninety per cent of the customers positioned themselves directly at the bar counter and turned straight towards the counter or a member of staff. The research team found that visitors who do not wish to place an order would instinctively avoid these behaviours. Subconsciously, they maintain a small distance to the bar and turn away from it, e.g. when chatting to friends. ‘Effectively, the customers identify themselves as ordering and non-ordering people through their behaviour,’ says psychologist Dr. Sebastian Loth, one of the authors of the study.
The findings were used to update James’ programming. ‘With the update, James only talks to people whose position and body posture clearly indicate that they wish to order a drink,’ says Loth. Only if the system is certain that the customers would like to order a drink, James will respond in Received Pronunciation: ‘How can I help you?’ Cutting the queue is a no-go with James. It maintains the order of the customers by memorising who came first.
The project James is funded by the EU. It started in early 2011 and ends in January 2014. University of Edinburgh (United Kingdom) coordinates the project partners. The robot’s hardware is maintained and programmed by fortiss GmbH in Munich (Germany). Other partners in the project are Heriot-Watt University (United Kingdom) and the Foundation for Research and Technology-Hellas (Greece).
Original publication:
Sebastian Loth, Kerstin Huth and Jan P. De Ruiter: Automatic detection of service initiation signals used in bars, Frontiers in Psychology, http://dx.doi.org/10.3389/fpsyg.2013.00557, published online on 30 August 2013
Further information is available online at:
www.james-project.eu
