Listen Before You Speak: Response Planning from Listener Facial Reactions for Conversational Speech Generation

Paper recorded by Signals 4 on 2026-09-18 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-18 on arXiv · recorded by Signals 4 on 2026-09-21

Category: cs.CL · 自然语言处理 · first seen 2026-09-21

Abstract

Conversational speech depends on dialogue context and the listener's immediately preceding behavior. We propose ReACT-TTS, a two-stage framework that uses a one-second pre-response listener facial sequence to plan the next utterance's emotion and prosody before speech realization. On a strict dyadic MELD protocol, Temporal conditioning yields higher mean macro-F1 and VAD concordance than Text-only

Read on arXiv →