LEONARDO, Vol. 32, No. 5, pp. 405–411, 1999 405
Abstract
In this article, the author discusses the
inspiration, concept, and technology behind her
sound performance work using eye movements in
relationship to current research on human eye
movement. She also compares the playing of the
eye-tracking instrument to research on musical
improvisation using unconventional musical
instruments and “active music.”
Eye Movement
Non oculi tacuere tui.
Your eyes were not silent.
—Ovid [1].
Seeing is active. Vision cannot occur without finely tuned
movements of the eye, taking in patterns of light and color on the
retina which the mind must then translate into a coherent world.
The demand for realistic computer simulations and accurate
machine vision has sparked a renewed interest in human vision
among researchers of many disciplines. I would like to discuss
some of that research and how it relates to my own work with
vision-tracking musical instruments.
Eye movements can be divided into three distinct types that are
under voluntary control: convergence, smooth pursuit, and saccades.
Convergence is the process of following objects as they approach or
move away. Smooth eye movements keep the line of sight on a
selected object and compensate for motion on the retina that
might be caused by motion either of the object or of the head or
body. In order to be initiated or sustained, smooth pursuit requires
an external moving visual signal. T he third type of movement, sac-
cades , are those which interested me most for performance. T hese
movements are rapid jumps of the eye used to shift gaze to a cho-
sen object. Once a saccadic movement has been initiated, the move-
ment cannot be interfered with until it reaches completion [2].
Smooth eye movements and fixation occur in the intervals
between saccades. Intervals between saccades can be as long as sev-
eral seconds during steady fixation, and, in reading, about three
times per second. Even when fixating, the eyes continue to move.
They tend to drift and flick involuntarily and to oscillate back and
forth continuously, although these movements are extremely small
[3]. Sequences of fixation and saccades have been used to study
cognitive processing in the coordination of eye and arm move-
ments, during visual problem-solving tasks, and during reading [4].
An increase in the speed of saccades can be learned or taught
with daily practice, and although saccades cannot be interrupted
once they are initiated, many researchers indicate that saccades are
planned, controllable activities. Some researchers suggest that shift-
ing attention to another location while the eye remains stationary
is the same as planning a saccadic eye movement. In other words,
sequences of saccades and fixation are directly related to attention.
Attention is allocated to a target shortly before the saccade is made
to look at it [5].
Allocation of visual attention is related to the content and mean-
ing of the subject. Saccadic eye movements in particular are used to
inspect a visual scene, requiring the integration of discrete time
frames into a stable picture of the scene. In 1967, A.L. Yarbus con-
ducted a series of experiments in which recordings of eye movements
made while looking at various paintings show systematic preferences
in movements: the eyes would repeatedly look at those elements that
would seem to be most relevant to the painting’s content [6].
Active Vision
Controlling Sound with Eye Movements
Andrea Polli
© 1999 Andrea Polli, received 25 January 1999
Andrea Polli, 1325 S. Wabash, #102, Chicago, IL 60603, U.S.A.
E-mail: apolli@interaccess.com.
Web site: homepage.interaccess.com/~apolli/