Tracking the Authentic and In-the-wild Emotions Using Speech

This first-of-its-kind study aims to track authentic affect representations in-the-wild. We use the ‘Graz Real-life Affect in the Street and Supermarket (GRAS²)’ corpus featuring audiovisual recordings of random participants in non-laboratory conditions. The participants were initially unaware of being recorded. This paradigm enabled us to use a collection of a wide range of authentic, spontaneous and natural affective behaviours. Six raters annotated twenty-eight conversations averaging 2.5 minutes in duration, tracking the arousal and valence levels of the participants. We generate the gold standards through a novel robust Evaluator Weighted Estimator (EWE) formulation. We train Support Vector Regressors (SVR) and Recurrent Neural Networks (RNN) with the low-level-descriptors (LLDs) of the ComParE feature-set in different derived representations including bag-of-audio-words. Despite the challenging nature of this database, a fusion system achieved a highly promising concordance correlation coefficient (CCC) of .372 for arousal dimension, while RNNs achieved a top CCC of .223 in predicting valence, using a bag-of-features representation.