14 th International Symposium on Particle Image Velocimetry – ISPIV 2021 August 1–5, 2021 Investigating Optimal Training and Uncertainty Quantification for CNN-based Optical Flow D. Kurihara 1 , G. Blois 1 , H. Sakaue 1 , D.E. Schiavazzi 1,2 1 University of Notre Dame, Department of Aerospace and Mechanical Engineering, Notre Dame, USA 2 University of Notre Dame, Department of Applied and Computational Mathematics and Statistics, Notre Dame, USA 1 Introduction Optical Flow (OF) techniques provide “dense estimation” flow maps (i.e. pixel-level resolution) of time- correlated images and thus are appealing to applications requiring high spatial resolutions. OF methods revolve around mathematical descriptions of the image as a collection of features, in which the pixel-level light intensity is the primary variable (Horn and Schunck, 1981). Feature tracking often involves the notion of scale invariance. Traditional OF approaches, merely based on mathematical formulations, have suffered from many challenges, especially when directly applied to images of fluid flows textured with tracer particles (hereafter PIV-like images). Due to the limited number of computationally manageable features and sub- optimal regularization methods, successful implementation of past approaches has been limited to highly textured images and small displacement dynamic ranges. Recent deep learning-based methods have effectively removed several limitations, offering an oppor- tunity to revitalize OF techniques. Being these based on structural features, the natural ability of artificial neural networks (ANN) with deep architectures to extract a large collection of such features at multiple resolutions through convolutions with learnable kernels can be brought to bear. In this context, a number of newly proposed Convolutional Neural Networks (CNNs), originally developed for computer vision applica- tions, have been successfully applied to the estimation of flows from dynamic scenes with moving objects. A recent example is LiteFlowNet (Fig. 1(a)), which provides state-of-the-art performance on a number of datasets including rigid body motion of objects in space, vehicles moving in traffic and, most notably, com- puter animated sequences (Hui et al., 2018). However, there is still limited understanding of which network setup (e.g. architecture, hyperparameter selection, etc.) provides optimal flow accuracy for PIV-like images. In this context, the goal of this study is to 1) provide a quantitative understanding of how LiteFlowNet performs for PIV-like images under different training paradigms and hyperparameter setups, and also to 2) extended the capabilities of LiteFlowNet to quantify flow uncertainty. 2 Methods LiteFlowNet is a family of recently proposed deep neural networks for optical flow estimation (Hui et al., 2018, 2020). Its architecture consists of pyramidal feature extraction (NetC) followed by a cascade of flow inference modules (NetE), i.e., a matching module, a subpixel module, and a regularization module. The matching module identifies the corresponding features in the image pair, the subpixel module corrects the flow estimation and provides subpixel accuracy, whereas the regularization module is used to refine the flow estimate near the boundary of moving objects. The network progressively identifies flow features in a coarse-to-fine resolution pipeline, starting from level 6 (coarser resolution) to level 2 (finer resolution). The original LiteFlowNet was trained using a staged process (see Fig. 1(a)), and did not support quantification of flow uncertainty. The first portion of this work assesses the predictive performance of LiteFlowNet using available pre-trained weights. Tests targeted synthetic PIV-like image sets with varying particle density, size and displacement. We then explored the potential of LiteFlowNet when trained from PIV-specific examples, following two different training paradigms proposed in the literature. In addition, we augmented the Lite- FlowNet architecture with dropout layers providing both a regularization mechanism and the possibility to estimate uncertainty from prediction ensembles.