Vis Comput DOI 10.1007/s00371-014-0986-6 ORIGINAL ARTICLE Fast Bokeh effects using low-rank linear filters Tim McGraw © Springer-Verlag Berlin Heidelberg 2014 Abstract We present a method for faster and more flexi- ble approximation of camera defocus effects given a focused image of a virtual scene and depth map. Our method lever- ages the advantages of low-rank linear filtering by reducing the problem of 2D convolution to multiple 1D convolutions, which significantly reduces the computational complexity of the filtering operation. In the case of rank 1 filters (e.g., the box filter and Gaussian filter), the kernel is described as ‘sep- arable’ since it can be implemented as a horizontal 1D con- volution followed by a 1D vertical convolution. While many filter kernels which result in bokeh effects cannot be approx- imated closely by separable kernels, they can be effectively approximated by low-rank kernels. We demonstrate the speed and flexibility of low-rank filters by applying them to image blurring, tilt-shift postprocessing, and depth-of-field simu- lation, and also analyze the approximation error for several aperture shapes. Keywords Bokeh · Blur · Filter · Depth-of-field 1 Introduction ‘Bokeh’ (from the Japanese ‘boke’) refers to the visual qual- ity of the blurry, and out-of-focus areas in photographs [18]. Bokeh effects are caused by multiple light rays from a sin- gle point in a scene passing through a camera aperture and striking different points on the image plane. Very unfocused regions of the image may be characterized by a distinctive pattern of highlights roughly the shape of the camera aper- ture. This can be used for artistic effects in photography when T. McGraw (B ) Purdue University, West Lafayette, USA e-mail: tmcgraw@purdue.edu using lenses that have a shallow depth-of-field (i.e., a narrow range of distances in which objects are in focus). For imag- ing systems that approximate a pinhole camera, the whole image will be in focus and bokeh effects will be absent. However, bokeh effects can be added as a post processing step. This is the approach used in real-time graphics because images generated by polygon rasterization appear to be in focus everywhere. The lens elements in a camera are designed to focus rays of light originating near the focal plane so that they converge on the image plane (see Fig. 1a). Rays originating far from the focal plane may be spread into a circle of confusion (CoC) on the image plane (see Fig. 1b). The aperture of the lens, formed by a number of pivoting blades, opens and closes to control the amount of light enter- ing the camera (Fig. 1c). The bokeh pattern is formed by the bundle of light rays that pass through the aperture. The shape of the bokeh is influenced by the number of blades, whether the blades are straight or curved, and the size of the aperture opening. Variations of light intensity within the bokeh are influenced by the optical properties of the lens elements. The basic approach to simulating the bokeh effect is to integrate over the contributions of the many light rays that may be incident at each point on the image plane. Our approach is an image space technique which requires only a single image and depth map. Unlike other techniques based on separable filters which make restrictive assumptions about the nature of the bokeh, our method can simulate the effect of a wide variety of apertures and lenses. Some other methods based on stochastic sampling permit a more general model- ing of the bokeh shape, but work with a sparse set of samples that can lead to noise artifacts. Our method produces images with no such noise artifact. Using low-rank linear filters, we can combine the computational benefits of separable filters with the flexibility of stochastic sampling. 123