IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 36, NO. 5, SEPTEMBER 2006 681 Automatic Detection and Elimination of Specular Reflectance in Color Images by Means of MS Diagram and Vector Connected Filters Francisco Ortiz and Fernando Torres, Member, IEEE Abstract—This paper proposes a new method for the detection and elimination of specular reflectance in color images of real scenes. We use a two-dimensional histogram that allows us to relate the signals of intensity and saturation of a color image, and to identify the specularities in an area of the histogram. This is known as the Intensity-Saturation (MS) diagram, and it is constructed from the Intensity-Saturation-Hue (MSH) generalized color space. An experimental and detailed study of the presence of specularities in the MS diagram for different types of materials in real scenes is carried out. To eliminate the specularities detected, we use a new connected vectorial filter based on the extension of mathematical morphology to color images, employing a lexicographical order. This new filter operates only in the bright areas previously detected, avoiding the high cost of processing the connected filters and the related oversimplification. The proposed method achieves results similar to current methods, but without the need for costly multiple-view systems or stereo images. Index Terms—Brightness detection, brightness elimination, color mathematical morphology, connected vectorial filters, Intensity-Saturation (MS) diagram. I. INTRODUCTION I N INDUSTRIAL visual inspection systems, images are ac- quired in work environments where the illumination plays an important role. Sometimes a bad adjustment of illumination can introduce brightness (highlights or specular reflectance) in the objects captured by the vision system. The presence of such brightness alters the pattern recognition process because the pre- vious stage of detection of edges in the objects fails. As such, the correct identification of objects, a goal in computer vision, is difficult to realize. The elimination of specular reflections is not only interesting in visual inspection, but also in other fields of computer vision, restoration, and reproduction of images, since the brightness affects the visual quality of the scene. There is no commercial software application that allows the automatic elimination of such specularities. To be able to attenuate the effect of the specular reflectance in the captured scene, the phenomenon in the image must first be detected and identified. The dichromatic reflection model proposed by Safer [1] is a tool that has been used in many meth- Manuscript received March 1, 2004; revised September 15, 2005. This paper was recommended by Guest Editors J. Salvador S´ anchez, F. Pla, and D. Maravall. The authors are with the Automatics, Robotics and Computer Vision Group, Department of Physics, Systems Engineering and Signal Theory, University of Alicante, 03690 Alicante, Spain (e-mail: fortiz@ua.es; fernando.torres@ua.es). Digital Object Identifier 10.1109/TSMCC.2005.855424 ods for detecting specularities. Such a model supposes that the interaction between the light and a dielectric material produces different spectral distributions in the object; i.e., the specular and diffuse reflectances. Diffuse component is a product of il- lumination and surface pigments, whereas specular reflectance has the same spectral makeup as its incident illuminant. The color of a given pixel in an image is a linear combination of a function of diffuse reflection and a function of specular reflection. Based on this model, Lin et al. [2] have developed a system for eliminating specularities in image sequences by means of stereo correspondence. Bajcsy et al. [3] use a chromatic space based on polar coordinates that allows the detection of specular and diffuse reflections by means of the previous knowledge of the captured scene. Klinker et al. [4] employ a pixel clustering algo- rithm which has been shown to work well in detecting brightness in images of plastic objects. Gershon et al. [5] and Lee et al. [6] use chromatic information for highlight identification. A similar approach is presented by Sato and Ikeuchi [7], who employ a so-called temporal-color space to extract the specular reflection and the body reflection. These previous approaches have pro- duced good results, but they have requirements that limit their applicability such as the use of stereo or multiple-view systems, the previous knowledge of the scene, or the assumption of a homogeneous illumination, without considering the interreflec- tions present in most typical real scenes. In this paper, we develop an automatic system for the de- tection of highlights in images by means of the use of a two- dimensional (2-D) histogram of intensity and saturation signals from a three-dimensional (3-D) polar coordinate color repre- sentation. This representation allows us to obtain a specular reflectance map of the image. Once the specularities have been identified, they are eliminated from the image by applying a vectorial geodesic reconstruction algorithm, which has a low cost and avoids the oversimplification of the image. In this paper, the two most important steps of the method pro- posed for brightness detection and elimination are described. In Section II, we present the color space used for the processing, together with the MS diagram developed to detect the specular reflectance. In addition, experimental results for the detection of brightness in real scenes are given. In Section III, we present the extension of the geodesic operations to color images. In Sec- tion IV, we develop a new connected vectorial filter for elim- inating the highlights detected in Section II. This elimination and the parameters of the algorithm are presented in Section V. Finally, our conclusions are outlined in Section VI. 1094-6977/$20.00 © 2006 IEEE