Toward Adaptive Generation of Backchannels for Attentive Listening Agents Tatsuya Kawahara, Miki Uesato, Koichiro Yoshino, Katsuya Takanashi Abstract Backchannels play an important role in spoken dialogue, especially in at- tentive listening such as counseling. Appropriately coordinated backchannels help establish rapport in that kind of dialogue. We investigate whether and how syn- chrony is expressed by the prosodic features of backchannels with respect to the preceding speaker’s utterances. By analyzing counseling dialogue, we find out cor- relation patterns according to the type of backchannels and prosodic features; a larger correlation is observed for reactive tokens than acknowledging tokens and for the power features than the pitch features. Based on the observations, we also conduct prediction of prosodic parameters of backchannels in order to replace the conventional conversational systems that generate monotonous backchannels. 1 Introduction Feedback behaviors play an important role in smooth communication [1]. In speech communication or spoken dialogue, verbal backchannels, such as “okay” and “right” in English, convey feedback. Without the feedback, the speaker would be anxious whether the communication is well maintained and would feel as if talking to a “machine”. Backchannels are used to express the listener’s feedback to what is uttered while suggesting that the current speaker can keep the dialogue turn. Specifically, backchannels express that the listener is listening, understanding, and agreeing to the speaker. Backchannels can also be used to express the listener’s assessment such as surprise, interest and sympathy. The variety of these roles is correlated with lexi- cal and prosodic patterns of the backchannels [2, 3]. In addition to the effect of individual backchannels, backchannels make a “rhythm” of the dialogue as a whole. By making “synchrony”, dialogue partners feel comfortable in keeping the dialogue. The phenomenon is regarded as one aspect of All authors are with School of Informatics, Kyoto University, Sakyo-ku, Kyoto 606-8501, Japan. 1