Int J Soc Robot (2013) 5:35–51
DOI 10.1007/s12369-012-0169-4
Perception and Generation of Affective Hand Movements
Ali-Akbar Samadani · Eric Kubica · Rob Gorbet ·
Dana Kuli´ c
Accepted: 14 September 2012 / Published online: 10 October 2012
© Springer Science+Business Media Dordrecht 2012
Abstract Perception and generation of affective move-
ments are essential for achieving the expressivity required
for a fully engaging human-machine interaction. This pa-
per develops a computational model for recognizing and
generating affective hand movements for display on an-
thropomorphic and non-anthropomorphic structures. First,
time-series features of these movements are aligned and
converted to fixed-length vectors using piece-wise linear
re-sampling. Next, a feature transformation best capable of
discriminating between the affective movements is obtained
using functional principal component analysis (FPCA). The
resulting low-dimensional feature transformation is used for
classification and regeneration. A dataset consisting of one
movement type, closing and opening the hand, is considered
for this study. Three different expressions, sadness, happi-
ness and anger, were conveyed by a demonstrator through
the same general movement. The performance of the de-
A. Samadani ( ) · D. Kuli´ c
Department of Electrical and Computer Engineering, University
of Waterloo, 200 University Avenue West, Waterloo, Ontario,
N2L 3G1, Canada
e-mail: asamadan@ecemail.uwaterloo.ca
D. Kuli´ c
e-mail: dkulic@ece.uwaterloo.ca
E. Kubica
Department of Systems Design Engineering, University of
Waterloo, 200 University Avenue West, Waterloo, Ontario,
N2L 3G1, Canada
e-mail: eric.kubica@uwaterloo.ca
R. Gorbet
Center for Knowledge Integration, University of Waterloo,
200 University Avenue West, Waterloo, Ontario, N2L 3G1,
Canada
e-mail: rbgorbet@uwaterloo.ca
veloped model is evaluated objectively using leave-one-out
cross validation and subjectively through a user study, where
participants evaluated the regenerated affective movements
as well as the original affective movements reproduced both
on a human-like model and a non-anthropomorphic struc-
ture. The proposed approach achieves zero leave-one-out
cross validation errors, on both the training and testing sets.
No significant difference is observed between participants’
evaluation of the regenerated movements as compared to the
original movement, which confirms successful regeneration
of the affective movement. Furthermore, a significant effect
of structure on the perception of affective movements is ob-
served.
Keywords Affective hand movements · Affective
computing · Functional data analysis · Feature extraction ·
Human affect perception · Affective movement generation
1 Introduction
Humans are adept at perceiving and estimating the affective
states of others [77]. One important channel of affect com-
munication is motor behavior (e.g., body and hand move-
ment). Studies in psychology suggest that body movements
and posture are important observable features of underly-
ing affective states [13, 14, 78]. Humans associate different
body movements and postures with distinct affective expres-
sions (e.g., anger is associated with frequent tempo changes)
[2, 16, 30, 52, 82], and are able to identify the feeling en-
coded in a displayed movement even when the demonstrator
tries to conceal its expression (e.g., negative body language)
[8, 22, 55]. Moreover, the psychology literature reports on
the human tendency to ascribe human-like social and affec-
tive attributes to non-anthropomorphic structures such as ab-