Geometric Image Parsing in Man-Made Environments Olga Barinova 1,⋆ , Victor Lempitsky 2 , Elena Tretiak 1 , and Pushmeet Kohli 3 1 Moscow State University 2 University of Oxford 3 Microsoft Research Cambridge Abstract. We present a new parsing framework for the line-based geo- metric analysis of a single image coming from a man-made environment. This parsing framework models the scene as a composition of geomet- ric primitives spanning different layers from low level (edges) through mid-level (lines and vanishing points) to high level (the zenith and the horizon). The inference in such a model thus jointly and simultaneously estimates a) the grouping of edges into the straight lines, b) the grouping of lines into parallel families, and c) the positioning of the horizon and the zenith in the image. Such a unified treatment means that the un- certainty information propagates between the layers of the model. This is in contrast to most previous approaches to the same problem, which either ignore the middle levels (lines) all together, or use the bottom-up step-by-step pipeline. For the evaluation, we consider a publicly available York Urban dataset of “Manhattan” scenes, and also introduce a new, harder dataset of 103 urban outdoor images containing many non-Manhattan scenes. The comparative evaluation for the horizon estimation task demonstrate higher accuracy and robustness attained by our method when compared to the current state-of-the-art approaches. 1 Introduction Recent years have seen a growing interest in the geometric analysis of a scene based on as little as a single image of this scene. Often the image of interest comes from a man-made environment, e.g. when the image is taken indoors or on a city street. In this case, the image is highly likely to contain a certain number of straight lines, which can be identified in the edgemap of the image, and which often can be fur- ther grouped into parallel families. The presence of such lines and their parallelism are known to be valuable cues for the geometric analysis. When a family of parallel lines is projected on the image, their projections are known to intersect in a single point in the image plane called vanishing point. The vanishing point uniquely characterizes the 3D direction of those lines (given ⋆ The first three authors are supported by Microsoft Research programs in Russia. Victor Lempitsky is also supported by EU under ERC grant VisRec no. 228180. K. Daniilidis, P. Maragos, N. Paragios (Eds.): ECCV 2010, Part II, LNCS 6312, pp. 57–70, 2010. c  Springer-Verlag Berlin Heidelberg 2010