His dialogue system not only recognises the user’s emotions but also can react to them.
Computer-based dialogue systems communicate with humans and are used to provide information, for example spoken travel information. ‘Emotions are an important component of human interaction,’ Dr Bui said, explaining his aims. ‘The motivation for my work was to improve communication between professional workers in crisis management scenarios. The problem with human emotions is that they are often difficult to interpret, which is especially so for a computer. Raising one’s voice can, for example, indicate enthusiasm, but it can also signal anger. We require extra information to be sure which of the two emotions is present. Human beings are trained to combine various types of information (which may be quite vague) and still be able to draw the correct conclusions. However, dealing with uncertainties is difficult to programme into computer software.’
Unlike others, Dr Bui’s dialogue system, the DDN-POMDP, can take emotions into account. The name is a combination of two techniques he uses: Dynamic Decision Network (DDN) and Partially Observable Markov Decision Process (POMDP).
During a test run, he applied the system to a navigation system for emergency services, to take into account the stress experienced by the user. The navigation system received input from a separate stress module that measured an emergency worker’s stress levels, taking these into account when the user communicated with the system. Whenever the user’s stress levels became raised, the system anticipated that the user was more likely to make mistakes, for example, and for that reason would request confirmation more often.
Dr Bui also foresees other applications for his technology: ‘I think DDN-POMDP could be of help wherever psychological stress obstructs an adequate evaluation of a crisis situation, e.g. it could be applied in emergency call systems or in tutoring programmes for rescue worker trainees.’
* Online demonstration: http://hmi.ewi.utwente.nl/icis/demonstrator/