PCA EVENT-BASED OPTICAL FLOW: A FAST AND ACCURATE 2D MOTION ESTIMATION Mahmoud Z. Khairallah Fabien Bonardi David Roussel Samia Bouchafa IBISC, Univ. Evry, Universit´ e Paris-Saclay, 91025, Evry, France ABSTRACT For neuromorphic vision sensors such as event-based cam- eras, a paradigm shift is required to adapt optical flow esti- mation as it is critical for many applications. Regarding the costly computations, Principal Component Analysis (PCA) approach is adapted to the problem of event-based optical flow estimation. We propose different PCA regularization methods enhancing the optical flow estimation efficiently. Furthermore, we show that the variants of our proposed method, dedicated to real-time context, are about two times faster than state-of-the-art implementations while signifi- cantly improving optical flow accuracy. Index Terms— Optical flow, Event-based cameras, Prin- cipal component analysis, 2D motion estimation. 1 Introduction Optical, flow plays a vital role in estimating ego-motion in robotics [1], depth [2] and foreground-background segmen- tation [3]. Standard cameras provide full-frame images at a fixed frequency resulting in data redundancy, low dynamic range, transmission latency and motion blur, which is not suit- able for proper optical flow estimation. Neuromorphic vision sensors [4] mimic the biological retina with very high tem- poral resolution and expanded dynamic range. Consequently, motion blur in standard frame-based cameras is omitted as well as high power consumption. In this regard, neuromor- phic vision sensors (also known as Event-based cameras) of- fer a better alternative to standard cameras [5]. Because of the ability to reach a sub-microsecond resolution, neuromor- phic sensors provide a large amount of data to be processed in short time intervals, and this requires algorithms that run fast while being robust to noise. The work present in the state- of-the-art is always a trade-off between better accuracy and faster computation. In this paper, we are concerned with the possibility to provide acceptable accuracy in different scenar- ios while maintaining low computational power so that it can be integrated into other schemes without hindering real-time applicability. The paper is organized as follows: Related work is introduced in section (2) and section (3) presents the specific properties of event-based cameras. Section (4) demonstrates the concept of Principal Component Analysis (PCA) and the benefits from its adaptation to an event-based optical flow scheme. Experi- mental setup and results are described in section (5). Section (6) summarizes the work presented in this paper. 2 Related Work Optical flow algorithms can be categorized into three groups categorized as non-optimization-based, optimization- based and neural-based groups. For the first group, in [6], events count is considered as intensity-equivalent values and used in a Lucas-Kanade event-based scheme [7]. Benos- man et al. [8] proposed a better representation of event-based optical flow that models created events as planes in a spatio- temporal neighborhood to estimate optical flow. Mueggler et al. [9] improved this scheme using RANSAC and optimiza- tion for refined lifetime estimation. The second group, Stoffregen et al. [10] estimate a vector that maximizes the variance of events as optical flow. Bar- dow et al. [11] introduce a joint estimation to provide dense grayscale intensity and optical flow using a GPU. Pan et al. [12] introduce another dense optical flow estimation using variational optimization. For the final group, one of the first attempts to use neu- ral networks in event-based vision is presented in [13]. It uses Spiking Neural Networks (SNN) [14] instead of Con- volutional Neural Networks to estimate discretized optical flow. A better adaptation of SNN is introduced in [15] where three-channel images of positive, negative event counts and timestamps are used to estimate dense optical flow, depth and ego-motion. Spike-FlowNet [16] estimates the optical flow at fixed frequency using events provided between two image frames. No absolute computational power requirements are disclosed in these articles. 3 Event-Based Nature Event-based cameras feature pixels that operate asyn- chronously and independently, hence, respond to luminosity change and trigger events whenever a threshold is attained. The nature of the event-based cameras results in a very high temporal resolution, where many events may be triggered in a microsecond. Each event is encoded as a tuple ⟨x, y, t, p⟩ where x and y are the pixel coordinates, t is the timestamp of the triggered event and p its polarity in {1, −1} correspond- ing to positive or negative luminosity change respectively. An event is triggered whenever a change exceeds a certain 3521 978-1-6654-9620-9/22/$31.00 ©2022 IEEE ICIP 2022 2022 IEEE International Conference on Image Processing (ICIP) | 978-1-6654-9620-9/22/$31.00 ©2022 IEEE | DOI: 10.1109/ICIP46576.2022.9897875