Scroll to top

Multimodal Emotion Recognition among Couples in Daily Life using Smartwatches

OCTOBER 7, 2022

George Jojo Boateng, MS
PhD Candidate and Doctoral Researcher
ETH Zurich

About the Presentation: Couples generally manage chronic diseases together and the management takes an emotional toll on both patients and their romantic partners. Consequently, recognizing the emotions of each partner in daily life could provide an insight into their emotional well-being in chronic disease management. Currently, the process of assessing each partner’s emotions is manual, time-intensive, and costly. Despite the existence of works on emotion recognition among couples, they are focused on English-speaking couples in the U.S. and none of those works have used data collected from couples’ interactions in daily life.

In this talk, I will present our work in which we developed a smartwatch-based machine learning system to recognize the emotions of German-speaking, Swiss-based romantic partners (managing type 2 diabetes) using 85 hours of sensor and self-reported data collected from everyday life. Our work contributes toward building automated emotion recognition systems that would eventually enable partners to monitor their emotions in daily life and enable the delivery of interventions to improve their emotional well-being.

About the Presenter:  George is a PhD Candidate and Doctoral Researcher in Applied Machine Learning at ETH Zurich, Switzerland, and a Visiting Researcher at the University of Cambridge, U.K. He is also the CEO and Cofounder of, an AI EdTech start-up that is democratizing science and technology education across Africa using smartphones and AI. He previously worked as an Applied Scientist at Amazon (Alexa AI) and he was recently recognized as one of the 2022 IBM New Creators and 2021 MIT Technology Review’s 35 Innovators Under 35. George has a BA in Computer Science and an MS in Computer Engineering from Dartmouth College, U.S.