Steering wheel’s angle tracking from camera-car Rita Cucchiara Andrea Prati Francesca Vigetti Dipartimento di Ingegneria dell’Informazione Universit` a di Modena e Reggio Emilia Modena, Italy cucchiara.rita@unimore.it prati.andrea@unimore.it francesca.vigetti@libero.it Abstract This paper proposes a general-purpose method to track the steering wheel’s absolute angle by using a single camera vision system mounted inside the car. The ap- proach is based on the modeling of the motion of the steering wheel, as it appears perspectively distorted by the point of view of the un-calibrated camera. We mod- ified the Lucas-Kanade method for an approximatively rotational motion model in order to provide the detec- tion and tracking of significant features on the wheel. The experimental results are compared with ground- truthed data obtained with different types of sensors. 1 Introduction Real-time analysis of videos acquired from a camera mounted on a moving vehicle (namely camera-car) can be very attractive due to the large amount of visual in- formation that can be extracted both inside the vehicle (to assess the driver conditions and control the environ- ment to prevent from dangerous situations) and outside the vehicle (for automatic guidance purposes, as vehicle control and obstacle avoidance). In the first context, new research activities are devoted to the assessment of the driver’s posture for smart air bag deployment, or to the acquisition of driving information. Another example is the use of cameras to detect potentially dan- gerous situations in which the driver is distracted (e.g., because he responds to a cell phone while is driving). In this framework an interesting problem is the detec- tion of the steering-wheel rotation angle. The possibil- ity to compute this angle in real-time can be exploited to provide a feedback to the driver in terms of vir- tual (or augmented) reality, or to support an automatic guidance system, or to analyze the style of the driving by observing how the steering wheel’s angle changes along time. Theoretically, the same information could be obtained by other types of sensors, such as electro-mechanical sensors, potentiometers, and so on, applied to the steer- ing wheel. The advantages of a vision-based system are (a) (b) Figure 1: Examples of applications: (a) an application for automated telemetry from camera-car videos (with the courtesy of Ferrari Spa, Italy) and (b) a car testing facility (with the courtesy of Centro Ricerche Fiat - Orbassano, Italy). basically three: first, the other types of sensors require a more invasive installation, and, moreover, cameras can be easily moved from one vehicle to another; sec- ond, electronic sensors can not work on pre-registered data, i.e. they can only obtain results on the moment, in real-time; third, the amount and the semantics of the information provided by a camera are more than any other type of “blind” sensor. As an example, refer to Fig. 1(b) where a special steering wheel equipped with potentiometers is used to acquire ground-truthed data. According with these considerations, we propose a general-purpose approach to detect the rotation angle of the steering wheel in a reliable manner. The method can be used to track the trajectory of the car by track- ing the rotation angle frame by frame. research works address the problem of detecting the motion of rigid objects: often the motion is assumed as a translational model (for instance the Lucas-Kanade method and derived approaches [3]); more generally, an affine model in the 3D space is assumed: this ap- proach is very general and very complex and there- fore highly time consuming. Thus, is often used as a first qualitative step to detect motion in videos [2]. In this work, instead, we start from a constrained motion model since we aim to detect and track the rotation of