Chat bots know when you’re going to laugh
Soon, chat bots will be able to laugh at your jokes. They also know when you’re going to laugh – even before you are aware of this. This has been shown in a new thesis that investigated the gaze patterns linked to laughter.
‘Dialogue system’ refers to technologies such as chat bots or conversation agents that people can talk to. These systems use large amounts of data, machine learning and natural language processing to be able to imitate human interactions, and recognise speech and text inputs.
Things like rhetorical questions have long been a challenge in the field of computational linguistics when working with dialogue systems. Now a research group at the University of Gothenburg has discovered that laughter can be a key for dialogue systems to better understand intentions.
“Our studies have laid the foundation for integrating laughter into spoken dialogue systems,” says Vladislav Maraev.
He has just defended his thesis Who is laughing now? Laughter-infused dialogue systems and is part of the research group at CLASP, Centre for Linguistic Theory and Studies in Probability, at the University of Gothenburg.
“The thesis shows that neural networks are effective in predicting laughter from dialogue transcripts. In addition, we gave the same task to people, and it turned out that neural networks were much better at it than untrained people. This suggests that people don’t know why they laugh at particular junctures – even though their laughter follows consistent patterns.
Related to gaze patterns
For a dialogue system involving an avatar or an embodied conversation agent for example, it is important to learn how laughter is linked to other behaviours to be able to put system laughter in appropriate places with the aid of AI. The researchers therefore examined how laughter relates to gaze patterns when people interact with each other.
“We showed that the function of laughter is related to different gaze patterns, in particular that gaze can signal the onset and end of laughter. Another finding is that different types of laughter are accompanied by different gaze patterns in the person laughing as well as the person with whom they are interacting.
These findings suggest that future conversation agents should not handle all laughter in the same way.
“They should be sensitive to precise coordination between laughter and gaze, which is entirely dependent on the type of laughter involved ," says Vladislav Maraev.
The thesis also studies how laughter affects the interpretation of a user's communicative intentions. An example is rhetorical questions, which on the surface look like a question but where the intention is not to ask a question at all, and therefore no response is needed.
For computational linguistics applications (computer programs used to analyse, understand and generate natural language) it has long been a challenge to understand figurative meanings.
“We discovered that it is useful to use laughter for this purpose. It was fascinating to see that laughter helps to clarify meaning when the system is required to understand the user’s intention. This was true of both literal and figurative interpretations of utterances. We showed that laughter alone can reveal the communicative intention,” says Vladislav Maraev.
Humour as an interactive phenomenon
By drawing parallels between laughter and what is called ‘grounding’ in communication contexts – that is, signals that show that everyone in a conversation is on board with it, for example, by uttering an ‘mhm’ – the researchers looked at laughter that was used to respond in the negative to questions. This gave them an insight into the possibility of including knowledge of laughter in machine learning.
“This contribution to the research is an example of how laughter can be used in dialogue systems and shows the potential for expanding the research to other types of laughter, such as when it used as positive feedback on a joke.
Another goal of the studies in the thesis was to bridge the gap between laughter and humour. Although humour is not a prerequisite for laughter, it is closely connected, and Vladislav Maraev stresses the importance of analysing humour as an interactive phenomenon and how it is linked to laughter.
“We proposed a theory that we believe is the first to take an interactive approach to humour. We investigated how humour is related to reasoning about social conventions and other learned and implied assumptions and explained our theory on the basis of foundations that are external to existing theories of humour and used in discussions of other linguistic and interactive phenomena.
The thesis Who is laughing now? Laughter-infused dialogue systems was publicly defended on 29 August at the Faculty of Humanities, University of Gothenburg.
Link to the thesis: https://gupea.ub.gu.se/handle/2077/72030
Contact details:
Vladislav Maraev, phone: +46(0)31-786 2936, e mail: vladislav.maraev@gu.se
