7 Application-oriented Assessment of Computer Vision Algorithms Peter Klausmann 1 , Stefan Fries 1 , Dieter Willersinn 1 , Uwe Stilla 2 , and Ulrich Thönnessen 2 1 Fraunhofer-Institut für Informations- und Datenverarbeitung (IITB) Karlsruhe, Germany 2 Forschungsinstitut für Informationsverarbeitung und Mustererkennung Ettlingen, Germany 7.1 Introduction ................................ 133 7.2 Analytical versus empirical performance analysis ........ 134 7.2.1 Analytical performance characterization ........ 134 7.2.2 Empirical performance characterization ......... 135 7.3 Application-oriented empirical algorithm assessment ..... 136 7.3.1 Task, performance criteria and requirement profile . 137 7.3.2 Assessment data ........................ 138 7.3.3 Image data acquisition .................... 140 7.3.4 Provision of training data for development ....... 140 7.3.5 Algorithm performance characterization ........ 140 7.3.6 The assessment function ................... 141 7.3.7 Overview of the assessment procedure ......... 142 7.4 The assessment system ......................... 143 7.5 Example: Assessment of a detection algorithm ......... 145 7.6 Conclusion ................................. 149 7.7 References ................................. 149 7.1 Introduction The crucial question to be answered during the assessment of a com- puter vision algorithm is to what extent is the algorithm performance useful? The utility of an algorithm can only be stated with respect to an application, hence the assessment of computer vision algorithms 133 Handbook of Computer Vision and Applications Copyright © 1999 by Academic Press Volume 3 All rights of reproduction in any form reserved. Systems and Applications ISBN 0–12–379773-X/$30.00