Shading into Texture and Texture into Shading: an Active Approach Jean-Yves Herr6 and John (Yiannis) Aloimonos Computer Vision Laboratory Center for Automation Research University of Maryland, College Park, Md 20742 Abstract We present a theory, based on the concept of active vision, for the fusion of informa- tion provided by different "shape from x" algorithms (here shading and texture), and a method for choosing the observer's activity optimizing ~he recovery of ~he shape. 1 Introduction 1.1 The "shape from x" problem One of the main research themes in modern computer vision is the "shape from x" problem. Following the paradigm introduced by David Marr ([Marr82]), attempts have been made to study modules identified in the animal visual system which perform the extraction of the shape of observed objects using cues such as shading, texture, motion or stereo. The last few years have seen many models and algorithms being proposed, still the problem is far from solved (see [Horn86] or [Aloim88] for extensive description of th e state of the field). Even assuming we eventually dispose of satisfying algorithms, the problem of deciding where and in which conditions to apply them remains unexplored. We propose here a general method for unification of shading and texture through an active observer mad apply it to current models of these modules. The use of the resulting algorithm is shown to be appropriate regardless of the intensity distribution in the image. 1.2 The active observer The concept of Active Perception was recently introduced in [Bajcs86] and further an- alyzed in [Aloim87]. An active observer, when engaged in an activity (self motion, tracking, focusing,...), modifies the constraints underlying a given phenomenon (and the equations describing them) and thus creates new information that will help in elim- inating ambiguities, make the solution easier to get and often more reliable, that is, more robust. 2 Unification of the two modules The observer considered here is a monocular optical system (camera) we will represent by a classical pinhole model (optical center O, optical axis OZ, focal length f). A point M of the object projects in the image plane as m. Slightly abusing the notation, we will identify the coordinates vector M (resp. m) with the position vector ~ (resp ~m).