Artefacts in Image and Video Systems: Classification and Mitigation Amal Punchihewa and Donald G. Bailey Institute of Information Sciences & Technology, Massey University g.a.punchihewa@massey.ac.nz d.g.bailey@massey.ac.nz Abstract This paper reviews a range of image and video artefacts, their causes, and mitigation techniques. Due to processing and distortions in transmission, the final display of visual data may contain artefacts. These can be classified into four types based on their origin: capture, processing (coding and decoding), delivery and display. The most significant cause of artefacts is the limited availability of bandwidth, and the tradeoffs associated with using this optimally. Often objective measures of image quality are poorly correlated with perceived image quality. This makes the effectiveness of various mitigation techniques harder to quantify. Keywords: image, video, artefacts, classification, coding, multipath, mitigation 1 Introduction For the purpose of communication we need to acquire, process and deliver information, visual data, including images and video. In a multimedia environment, an image can be described as a two- dimensional representation of a scene or other visual data. Video can simply be treated as a sequence of images but often contains additional information such as timing or synchronization signals. Video also implies correlated audio, which is often included in the video stream. Audio artefacts are beyond the scope of this paper. Images and video are strong forms of information in comparison to other forms of information such as text, sound and signs. When an item of visual data is used for a given medium, it is processed based on the constraints of the medium. Since each image and video contains a large quantity of data, delivery generally requires a high bandwidth. A common model for this process is shown in Figure 1. Fortunately, most image data contains significant redundancy in the form of high correlation of adjacent pixel values, and between adjacent frames of video. 2 Artefacts When visual information is captured, processed and delivered to the final recipient, the displayed picture may differ from the original. Artefacts are any visible differences that are a direct result of some technical limitation at any stage of the communication process. Image and video artefacts can be broadly classified into four types based on their origin. They are due to capture, processing (coding and decoding), delivery and display. They occur in both analogue and digital systems however some artefacts may be more prevalent or visible in one type of system. 2.1 Capture Images and video are captured using cameras that comprise of an optical system and a sensor with processing circuitry. Artefacts based on capture will affect both analogue and digital systems as this is at the front end of the image acquisition. Reflected light from the object or scene forms an image on the sensor. Optical imperfections distort the image captured; limited depth of field can make parts of the image appear defocused; non- uniform magnification leads to barrel or pincushion distortion; chromatic aberrations introduces colour fringing; and vignetting- decreasing intensity towards image corners due to lenses being faster in centre than periphery. Capture artefacts may include visible effects due to interlaced scanning (see for example Figure 2), aliasing (both temporal and spatial), or distortion due to perspective. Area sampling on the sensor limits the resolution and can give rise to contrast inversions. With digital image capture, quantisation introduces additional noise, and can give contouring. Study of these artefacts and their mitigation is very important to avoid the propagation of artefacts. Figure 2: Interlace artefact from panning camera 2.2 Processing Once image or video data is captured it needs to be processed before delivery through the communication medium. This processing is required to meet constraints such as bandwidth limitations imposed by the medium and to provide immunity against medium noise. There are many coding techniques for removing the redundancies in images and video. Coding can introduce artefacts Figure 1. Image/Video processing model for communication Acquisition or Capture Signal Processing (Coding) Delivery Signal Processing (Decoding) Display