Robust incremental rectification of sport video sequences Jean-Bernard Hayet Justus Piater Jacques Verly Department of Electrical Engineering and Computer Science (Institut Montefiore) University of Li` ege Building B28, B-4000 Li` ege, Belgium FirstName.LastName@ulg.ac.be Abstract We describe an important element of an automatic sport analysis system. This element continuously estimates the image-to-model homography from the video stream of a single camera. Here, we focus on the incremental image-to-image updating of the homography matrix, which greatly facilitates real-time operation. This updating relies on the automatic tracking of interest lines and points and on their use for robust homography estimation. Results on real video sequences are shown. 1 Introduction Proceedings of the British Machine Vision Conference 2004, pp. 687–696 This work was sponsored by the R´ egion Wallone under DGTRE/WIST contract 031/5439. Sport broadcasters are beginning to rely on computer vision techniques to enhance their video productions. Examples are the smart overlay of advertisement on the grass without obscuring the players and the automatic analysis of games. This requires static (e.g., length) and dynamic (e.g., speed) metrics from one or more images produced by one or more cameras. With metric information, it becomes possible to rectify images, i.e., to project them onto a model of the game field, thereby facilitating subsequent analysis. Current commercial systems for automatic rectification are expensive. Indeed, most of them rely on sensors embedded in cameras. They measure all relevant camera parameters, such as orientation in space and zoom. In this paper, we focus on a subsystem of a fully-automated sport analysis system we are developing. This subsystem allows us to maintain a continuous estimate of the homographic mapping [4] between a model of the field and images of it, without hav- ing recourse to any kind of sensor on the cameras and/or the players. To achieve this functionality, we use three complementary strategies. The first strategy computes the homography from scratch. It is used initially and then periodically to reinitialize the system. The second strategy quickly computes the homog- raphy from a current estimate by maintaining image-to-model correspondences, thereby enabling real-time operation. The third strategy updates the homography incrementally from image-to-image correspondences. It is helpful when image-to-model matches are not sufficient. Drifts due to this last mode of operation can be compensated for by peri-