Automated Removal of Partial Occlusion Blur Scott McCloskey, Michael Langer, and Kaleem Siddiqi Centre for Intelligent Machines, McGill University {scott,langer,siddiqi}@cim.mcgill.ca Abstract. This paper presents a novel, automated method to remove partial occlusion from a single image. In particular, we are concerned with occlusions resulting from objects that fall on or near the lens during exposure. For each such foreground object, we segment the completely occluded region using a geometric flow. We then look outward from the region of complete occlusion at the segmentation boundary to estimate the width of the partially occluded region. Once the area of complete occlusion and width of the partially occluded region are known, the con- tribution of the foreground object can be removed. We present experi- mental results which demonstrate the ability of this method to remove partial occlusion with minimal user interaction. The result is an image with improved visibility in partially occluded regions, which may convey important information or simply improve the image’s aesthetics. 1 Introduction Partial occlusions arise in natural images when an occluding object falls nearer to the lens than the plane of focus. The occluding object will be blurred in pro- portion to its distance from the plane of focus, and contributes to the exposure of pixels that also record background objects. This sort of situation can arise, for example, when taking a photo through a small opening such as a cracked door, fence, or keyhole. If the opening is smaller than the lens aperture, some part of the door/fence will fall within the field of view, partially occluding the back- ground. This may also arise when a nearby object (such as the photographer’s finger, or a camera strap) accidentally falls within the lens’ field of view. Whatever its cause, the width of the partially-occluded region depends on the scene geometry and the camera settings. Primarily, the width increases with in- creasing aperture size (decreasing f -number), making partial occlusion a greater concern in low lighting situations that necessitate a larger aperture. Fig. 1 (left) shows an image with partial occlusion, which has three distinct regions: complete occlusion (outside the red contour), partial occlusion (between the green and red contours), and no occlusion (inside the green contour). As is the case in this example, the completely occluded region often has little high- frequency structure because of the severe blurring of objects far from the focal plane. In addition, the region of complete occlusion can be severely underexposed when the camera’s settings are chosen to properly expose the background. In [7], it was shown that it is possible to remove the partial occlusion when the location and width of the partially occluded region are found by a user. Because Y. Yagi et al. (Eds.): ACCV 2007, Part I, LNCS 4843, pp. 271–281, 2007. c  Springer-Verlag Berlin Heidelberg 2007