Subject: Computational Linguistics
Thesis title: Who is laughing now? Laughter-infused dialogue systems
Examining committee:
Professor Gabriel Skantze, Kungliga tekniska högskolan, Stockholm Doktor Magdalena Rychlowska, Queen’s University, Belfast
Professor Alexandra Weilenmann, Göteborgs universitet
Substitute if member in the committee will be missing:
Professor emeritus Torbjörn Lager, Göteborgs universitet
Opponent: Professor Matthew Stone, Rutgers School of Arts and Sciences
Chair: Doktor Ellen Breitholtz, Göteborgs universitet
Abstract
This thesis paves the way for including laughter in spoken dialogue systems in a domain-general and linguistically valid way using computational linguistics tools and methods. The thesis is concerned with three main areas.
The first area is the placement of laughter in relation to speech and other behaviours. We show that convolutional and recurrent neural networks can effectively predict laughs from dialogue transcripts, whereas human perforance in this task is significantly worse. Such prediction models allow dialogue systems to predict user laughter and, if needed, put system laughter in an appropriate place. Furthermore, we look at laughter placement in relation to gaze and show that laughter, performing different pragmatic functions, is related to different gaze patterns. These findings provide important implications for embodied conversational agents and social robots in regard to multimodal behaviour realisation and coordination.
The second area is concerned with interaction between laughter and the communicative intent of a user and system, as well as with the context in which the given intent occurs. We lay the groundwork for the central component of a spoken dialogue system by implementing a dialogue manager in a theoretically informed way using a proof-theoretic model based on linear logic. Our dialogue manager is then extended to support laughter functioning as feedback or a signal accompanying system feedback, and an answer to polar questions. Additionally, we look at how laughter can modify or form a dialogue act, and how laughter can improve Transformer-based deep learning models in the task of dialogue act recognition.
The third area is humour. Humour is intuitively related to laughter and some laughs highlight social norm violations and ironic statements, which bring laughter and humour closer together, even though humour is not necessary for laughter. We look at how humour is related to reasoning about social conventions and other learned and accommodated implicit assumptions, and how humour can be modelled in relation to creativity, implying situational and conversational creativity for artificial agents.
