A blind binaural real-time model for listening effort evaluated using continuous subjective listening effort rating

Martin Berdau, Daniel-José Alcala Padilla, Thomas Brand, Christian Rollwage, Jan Rennies

This study investigated real-time assessment and modeling of perceived listening effort (LE). The model consists of a binaural front-end, followed by a monaural back-end. As front-end, a novel blind real-time implementation of the binaural speech intelligibility model (BSIM) was developed, which models spatial release from masking by considering binaural unmasking and better-ear listening simultaneously. A neural network was used as back-end, which was trained on inputs and outputs of a LE prediction model based on phoneme classification called Listening Effort prediction from Acoustic Parameters (LEAP). A novel method for evaluating binaural real-time models of LE was developed, where simulated scenes with a target speaker and a noise interferer were used, which were either co-located or spatially separated. Dynamic changes were introduced to the scene by abruptly altering the signal-to-noise ratio and/or reverberation time. Participants continuously rated subjectively perceived LE using a slider interface with LE categories, while listening to the scenes via headphones. The model accurately predicted subjective LE, especially changes in signal-to-noise ratio and binaural benefits. It also predicted detrimental effects of reverberation as observed in the experiment, although the impact of reverberation was slightly overestimated. Human response times were estimated for further tweaking the model's integration time.