1 INTRODUCTION Nowadays the generation of a 3D model is achieved using range or image data or a combination of both methods. Range data contains already the three-dimensional coordinates necessary for the modeling phase while images needs a mathematical model to derive the object coordinates. Image-based approaches require information that can often be extracted automatically from the images. In some cases, automated procedures are not satisfactory or can not recover all the re- quired features, therefore manual or semi-automated measurements must be performed. After the measurements, the data must be structured and a consistent polygonal surface generated to build a realistic representation of the imaged scene. We can actually distinguish 4 alternative methods for scene modeling: • Image-based rendering (IBR): it does not include the generation of a geometric 3D model, but for particular objects and under specific camera motions and scene conditions, it might be considered a good technique for the generation of virtual views (Kang, 1999). The technique relies on either accurately knowing the camera positions or automatic ste- reo matching that, in the absence of geometric data, requires a large number of closely spaced images to succeed. Object occlusions and discontinuities particularly in large- scale and geometrically complex environments will also affect the output. The ability to move freely into the scene and viewing objects from any position may be limited depend- ing on the method used. It is therefore unlikely that IBR will be the approach of choice for purposes other than limited visualization. • Image-based modeling (IBM): it is the widely used method for geometric surfaces of ar- chitectural objects (Streilein 1994, Debevec et al. 1996, Van den Heuvel 1999, El-Hakim 2002) or for terrain and city modeling (Gruen 2000). In most of the cases, the most im- pressive and accurate results remain those achieved with interactive approaches. • Range-based modeling: this method can directly capture the 3D geometric information of an object. It is based on expensive active sensor (e.g. Breuckmann TM , Cyberware TM , ShapeGrabber TM ) that often lack in texture information. However, texture or color infor- Critical overview of image-based 3D modeling F. Remondino Institute of Geodesy and Photogrammetry, ETH Zurich, Switzerland S. El-Hakim Visual Information Technology (VIT), National Research Council, Ottawa, Canada ABSTRACT: In this paper we address the main problems and available solutions for the genera- tion of 3D models from terrestrial images. Close range photogrammetry deals since many years with manual or automatic image measurements for precise 3D modeling. Nowadays 3D scan- ners are also becoming a standard source for input data in many application areas, but image- based modeling still remain the most complete, cheap, portable, flexible and widely used ap- proach. After reviewing the different 3D shape techniques for surface reconstruction, we will report the full pipeline for 3D modeling from image data. Some modeling methods are also de- scribed and different examples are presented.