Dichoptic Fusion of Thermal and Intensified Imagery A. Toet, M.A. Hogervorst, M. van der Hoeven TNO Human Factors Kampweg 5 3769 DE Soesterberg, The Netherlands {lex.toet, maarten.hogervorst, marieke.vanderhoeven}@tno.nl Abstract - Subjects used the dichoptic combination of a monocular image intensifier (NVG) and a monocular uncooled microbolometer (LWIR) to detect and localise both visual targets and camouflaged thermal targets while moving through a dimly lit complex environment. The NVG imagery enabled the subjects to move freely through the environment with high accuracy, but did not mediate the detection of camouflaged thermal targets. The LWIR mode mediated the detection of camouflaged thermal targets but did not allow the detection of visual targets, and provided insufficient detail to allow accurate movement through the environment. Subjects were quite capable to dichoptically fuse the individual LWIR and NVG images, enabling them to detect all (visual and thermal) targets while moving accurately through the environment. We conclude that dichoptic fusion of NVG and LWIR imagery is quite feasible and is a simple way to provide observers with enhanced situational awareness in nighttime operations. Keywords: Image fusion, intensified imagery, NVG, LWIR, thermal imagery, dichoptic fusion. 1. Introduction Night vision devices provide the modern soldier with the ability to see, manoeuvre and shoot during periods of reduced visibility. Currently two types of night vision devices are in use: image intensifiers and thermal cameras. Image intensifiers capture ambient light (from the stars, moon or sky glow) and amplify it thousands of times by electronic means. In NVGs the image is shown via a phosphor display. The main advantage of NVGs are their small size, light weight, low power requirements and low cost. These properties have led to their widespread use. Thermal longwave infrared sensors detect the heat naturally radiated by all objects– including terrain, roads, buildings, vehicles, and people–and can distinguish differences in temperature between objects and their backgrounds. They provide the ability to detect targets at long range, and to look through smoke, dust, fog and other obscuring conditions. In contrast, image-intensifiers provide a short detection range and cannot see through smoke, dust, haze, and adverse weather at night. Recently small hand-held uncooled thermal cameras have become available, which operate on standard batteries. Because of their complementary nature, the combination of NVG and thermal imagery may significantly extend the range of conditions in which the dismounted soldier can operate. Compared to presently fielded NVGs, the addition of thermal imaging will provide the capability to see through battlefield obscurants (smoke screens) and to see in zero light conditions (enclosed spaces, buildings, caves). Fusion of thermal and intensified imagery can be achieved either optically or digitally. Digital image fusion has many advantages, since it enables the application of many different image enhancement techniques, the exchange of imagery with a command center and the inclusion of high level information from a battlefield management system. However, digital image processing requires significant power, reduces the NVG’s pixel count, still needs significant technology advances, and the associated human factors are not yet fully understood. In contrast, optical image fusion maintains the high resolution of the NVG imagery, requires no technology breakthroughs, and provide simple to operate designs for which the human factors are fully understood. As a result, optical image fusion based on designs that use simple and proven approaches and technologies is the most promising approach to the development of short term fieldable systems. Currently a team of ITT Industries/Raytheon is developing an Enhanced NVG (ENVG) for the US Army, which is based on a simple optical overlay of the video image from a miniature LWIR thermal (microbolometer) camera on to the image from a conventional direct-view night vision monocular, with the output from both channels passing through a beam combiner [3, 5]. This approach is relatively simple and inexpensive, making it particularly suitable for infantry head-mounted systems. However, it still has some disadvantages, including a mismatch of the overlay at the edge of the field of view, the use of a complex optical path, and the need to adjust the relative brightness of both images manually. The human brain has the remarkable capability to fuse the two disparate images from both eyes in realtime into a single consistent percept. This capability has been succesfully deployed to fuse multi-modal imagery in gaze-contingent displays [8]. Here we propose to combine NVG and thermal imagery by presenting the