Combined use of a priori data for fast system self-calibration of a non- rigid multi-camera fringe projection system Petros I. Stavroulakis* a , Shuxiao Chen a , Danny Sims-Waterhouse a , Samanta Piano a , Nicholas Southon a , Patrick Bointon a , Richard Leach a a Manufacturing Metrology Team, The University of Nottingham, UK, NG7 2RD ABSTRACT In non-rigid fringe projection 3D measurement systems, where either the camera or projector setup can change significantly between measurements or the object needs to be tracked, self-calibration has to be carried out frequently to keep the measurements accurate 1 . In fringe projection systems, it is common to use methods developed initially for photogrammetry for the calibration of the camera(s) in the system in terms of extrinsic and intrinsic parameters. To calibrate the projector(s) an extra correspondence between a pre-calibrated camera and an image created by the projector is performed. These recalibration steps are usually time consuming and involve the measurement of calibrated patterns on planes, before the actual object can continue to be measured after a motion of a camera or projector has been introduced in the setup and hence do not facilitate fast 3D measurement of objects when frequent experimental setup changes are necessary. By employing and combining a priori information via inverse rendering, on-board sensors, deep learning and leveraging a graphics processor unit (GPU), we assess a fine camera pose estimation method which is based on optimising the rendering of a model of a scene and the object to match the view from the camera. We find that the success of this calibration pipeline can be greatly improved by using adequate a priori information from the aforementioned sources. Keywords: Structured light scanning, calibration, fringe projection, convolutional neural network, information rich metrology, inverse rendering, photogrammetry 1. INTRODUCTION The concept of using a priori information about the measurement setup, or what we refer to as “information-rich metrology” (IRM), to improve the speed and/or accuracy of a measurement, is something which has been the focus of multiple research efforts in different measurement scenarios 2–5 . In this work we apply a priori information to improve the accuracy of inverse rendering as a method to quickly re-calibrate a flexible multi-camera 3D form measurement setup. A structured light system consists of light sources and cameras which are used to infer 3D information about an object. Irrespective of the methodology used (i.e. sinusoidal fringe projection, line scanning), in order to obtain accurate measurements of the object’s three-dimensional shape when using structured light systems, the camera(s) and light source(s) need to be calibrated in terms of their intrinsic parameters (i.e. lens aberrations, the optical centre, pixel sizes) and in terms of their location and pose with respect to the reference or ‘global’ coordinate system (also called the extrinsic parameters). The better the calibration of the cameras and light sources, the more accurate the measurement that is produced by the system. The camera calibration for structured light applications is normally performed by well-established methods, such as linear calibration 6 , the two-stage Tsai method 6 7 , vanishing points 8 and checkerboard plane technique (Zhang method) 6 9 . In this work we will use the versatile Zhang method available in Matlab to calibrate the cameras via a simple process. The most commonly used and widespread calibration methods which exist today for projector calibration 10–12 treat the projector as an inverse camera and hence use the same concept, as that applied to a binocular photogrammetry system, to calculate the homographies between a pre-calibrated camera and a projector. To achieve this, the projector either projects binary patterns onto the ‘square checkerboard’ pattern normally used to calibrate a camera 10 or multiple phase shifted patterns 11 to achieve homography between the projector pixels and the camera pixels. In this work, we used the method and software published by Moreno and Taubin 10 . Modeling Aspects in Optical Metrology VI, edited by Bernd Bodermann, Karsten Frenner, Richard M. Silver, Proc. of SPIE Vol. 10330, 1033006 · © 2017 SPIE CCC code: 0277-786X/17/$18 · doi: 10.1117/12.2269302 Proc. of SPIE Vol. 10330 1033006-1 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 06/28/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx