Comparametric HDR (High Dynamic Range) Imaging for Digital Eye Glass, Wearable Cameras, and Sousveillance Mir Adnan Ali, Tao Ai, Akshay Gill, Jose Emilio, Kalin Ovtcharov, and Steve Mann Department of Electrical and Computer Engineering University of Toronto 10 King’s College Rd, Toronto, ON, M5S3G4 Abstract—Wearable computing can be used to both extend the range of human perception, and to share sensory experiences with others. For this objective to be made practical, engineering considerations such as form factor, computational power, and power consumption are critical concerns. In this work, we consider the design of a low-power visual seeing aid, and how to implement computationally-intensive computational photography algorithms in a small form factor with low power consumption. We present realtime an FPGA-based HDR (High Dynamic Range) video processing and filtering by integrating tonal and spatial information obtained from multiple different exposures of the same subject matter. In this embodiment the system captures, in rapid succession, sets of three exposures, “dark”, “medium”, and “light”, over and over again, e.g. “dark”, “medium”, “light”, “dark”, “medium”, “light”, and so on, at 60 frames per second. These exposures are used to determine an estimate of the photoquantity every 1/60th of a second (each time a frame comes in, an estimate goes out). This allows us to build a seeing aid that helps people see better in high contrast scenes, for example, while welding, or in outdoor scenes, or scenes where a bright light is shining directly into the eyes of the wearer. Our system is suitable for being built into eyeglasses or small camera-based, lifeglogging, or gesture- sensing pendants, and other miniature wearable devices, with low-power and compact circuits that can be easily mounted on the body. I. I NTRODUCTION There are two broad categories of veillance, surveillance, and sousveillance. The primary dictionary definition, given in [1], of the word “surveillance” is surveillance, n. : a watch kept over a person, group, etc., especially over a suspect, prisoner, or the like: The suspects were under police surveillance. The etymology of this word is from the French word “surveiller” which means “to watch over”. Specifically, the word “surveillance” is formed from two parts: (1) the French prefix “sur” which means “over” or “from above”, and (2) the French verb “veiller” which means “to watch”. A more recently coined word is the word “sousveillance”, which is an etymologically correct opposite formed by re- placing the prefix “sur”, in “surveillance”, with its opposite, “sous”. Table I summarizes the veillances (surveillance and sousveillances) and the etymologies of these words. Sousveil- lance often refers to cameras borne by people, e.g. hand-held cameras or wearable cameras [2], [3], [4], [5], [6], [7]. TABLE I: The Veillances: Surveillance and Sousveillance English French to see voir to look (at) regarder to watch veiller watching (monitoring) veillance watching over (oversight) surveillance to oversee (to watch from above) surveiller over (from above) sur under (from below) sous “undersight” (to watch from below) sousveillance A. Motivation In traditional photography or surveillance, a great deal of effort is often expended to arrange the camera and lighting for best exposure. Photographers will often position themselves for best view, i.e. with the sun behind them, so that they are not shooting directly into the light. In our application, however, we wish to create a computer vision system (seeing aid) that can work in any lighting situation as might occur in day-to-day life. The Digital Eye Glass, for example, observes the real world in a similar fashion as the user observes (or would have observed, in the case of a blind individual). The sousveillance user’s total lack of control over lighting, combined with the wide variety of lighting situations encountered in the course of a day (e.g. from direct sunlight to candlelight) necessitate that any sousveillance apparatus be equipped to produce a useful signal, without requiring large battery packs or a large-aperture camera. B. Related Work Cameras can only take in photographs with limited dy- namic range. One method to overcome this is to combine differently exposed images of the same subject matter, pro- ducing a High Dynamic Range result [8], [9], [10]. HDR digital photography started almost 20 years ago. “The first report of digitally combining multiple pictures of the same scene to improve dynamic range appears to be Mann” [11]. Now, it is possible to produce HDR photography [12], [13] and video [14], [15], [16], [17] in realtime, on both high- power CPU/GPU systems, as well as low-power FPGA boards 107 978-1-4799-0929-2/13/$31.00 c 2013 IEEE