Geometry of imaginary spaces Jan J. Koenderink ⇑ Delft University of Technology, Man Machine Interaction Group, EEMCS, P.O. Box 5031, 2600 GA Delft, The Netherlands Katholieke Universiteit Leuven, Laboratorium voor Experimentele Psychologie, Tiensestraat 102, Bus 3711, 3000 Leuven, Belgium The Flemish Academic Centre for Science and the Arts, Academy Palace, Hertogsstraat 1, 1000 Brussel, Belgium article info Article history: Available online 25 November 2011 Keywords: Space Imaginary space Pictorial space Local sign External local sign Cues Natural perspective Depth abstract ‘‘Imaginary space’’ is a three-dimensional visual awareness that feels different from what you experience when you open your eyes in broad daylight. Imaginary spaces are experienced when you look ‘‘into’’ (as distinct from ‘‘at’’) a picture for instance. Empirical research suggests that imaginary spaces have a tight, coherent structure, that is very different from that of three-dimensional Euclidean space. This has to be due to some constraints on psychogenesis, that is the development of awareness. I focus on the topic of how, and where, the construction of such geometrical structures, that figure prominently in one’s aware- ness, is implemented in the brain. My overall conclusion—with notable exceptions—is that present day science has no clue. I indicate some possibly rewarding directions of research. Ó 2011 Elsevier Ltd. All rights reserved. 1. Natural perspective ‘‘Natural perspective,’’ or perspectiva naturalis, is best known from Euclid’s treatise (Burton, 1945) (Greek Optika; Latin: De aspectibus). It should be sharply distinguished from ‘‘painter’s per- spective,’’ or perspectiva artificialis, which plays no role in this paper, but became generically known as ‘‘perspective.’’ The latter involves ‘‘Alberti’s Window,’’ after Alberti’s (1435) Della pittura, and deals with the representation of the visual field on planar sur- faces. Unfortunately, these concepts are rarely distinguished. Here I interpret natural perspective in its original sense of ‘‘optics,’’ a proper subfield of physics. It deals with the potential of momentar- ily seeing things with a single, punctate eye, 1 and is thus to be con- sidered a form of information theory. 2 It has nothing to do with the transport of radiant power, thus the frequent discussions on Euclid’s use of the extramission theory are void (Koenderink, 1982). I recapitulate the basics of natural perspective here. Consider three-dimensional Euclidean space E 3 , augmented with a single ‘‘vantage point’’ O. Any point P 2 E 3  O is seen at a unique direction, called its ‘‘visual direction’’ with respect to the vantage point. Consider the ‘‘optic array’’ (Gibson, 1950) at the vantage point, which is simply the unit sphere S 2 , centered on O. Then the direction of P is conveniently labeled with its trace p 2 S 2 , where p =(P  O)/ kP  Ok (see Fig. 1). One has P = O + .p, where . is the ‘‘range’’ 3 of P with respect to the vantage point. Points with the same traces are seen in the same direction. When their ranges are different they are still distinct. I refer to such points as mutually ‘‘parallel.’’ 4 Berkeley (MDCCIX) famously de- clared that such parallel points cannot be distinguished on the basis of optics proper, because ranges are, in modern jargon, not ‘‘optically specified.’’ Slightly idealized, the human condition involves primarily day- light, clear air, and opaque rigid objects. Daylight is solar radiation in a narrow visual band centered at about 560 nm, often scattered by cloud covers. Clear air optically acts much like the vacuum, a perfect medium of the propagation of radiation. Opaque objects have surfaces that scatter radiation into all directions. This leads to the following basic laws of visual optics: I. You can see any object given unobstructed range, II. You cannot see objects that are occluded by other objects, and 0928-4257/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.jphysparis.2011.11.002 ⇑ Address: Delft University of Technology, Man Machine Interaction Group, EEMCS, P.O. Box 5031, 2600 GA Delft, The Netherlands. Tel.: +31 152784145; fax: +31 152787141. E-mail address: j.j.koenderink@tudelft.nl 1 In computer vision this is known as the ‘‘pinhole camera model.’’ In geometrical optics it is the center of the anterior nodal point of the optical system. In human vision it is most natural to use the center of rotation of the eye-ball. 2 Euclid uses the theory to account for visual acuity, by referring to a certain sparsity and thickness of rays. This can be related to the minimum étendue of about a wavelength squared of the wave theory of light. 3 ‘‘Range’’ is preferred over ‘‘distance,’’ because in the latter case one should not omit to say ‘‘from the eye.’’ 4 This usage is natural in the singly isotropic plane introduced later, where one has a perfect metrical duality between points and lines. Then it is natural to have both ‘‘parallel lines,’’ and ‘‘parallel points.’’ Journal of Physiology - Paris 106 (2012) 173–182 Contents lists available at SciVerse ScienceDirect Journal of Physiology - Paris journal homepage: www.elsevier.com/locate/jphysparis