Accepted by Measurement, 2013 A COMPARISON OF LEAST SQUARES AND MAXIMUM LIKELIHOOD METHODS USING SINE FITTING IN ADC TESTING Ján Šaliga , * István Kollár, Linus Michaeli, Ján Buša, Jozef Lipták, * Tamás Virosztek Technical University of Košice, Letná 9, 04120 Košice, Slovakia Email: {jan.saliga, linus.michaeli, jan.busa, jozef.liptak}@tuke.sk * Budapest University of Technology and Economics, Budapest, Hungary Email:{kollar,virosztek.tamas}@mit.bme.hu Abstract: ADC test methods require the best possible reconstruction of the input signal of the ADC under test from the acquired, therefore erroneous, ADC output data. The commonly used least squares (LS) fit and the recently introduced maximum likelihood (ML) estimation are competing methods. This paper presents a simulation-based comparative study of these estimation methods with the goal to investigate the behaviour of both methods and to determine their limits. Two alternative algorithms for the calculation of the maximum likelihood fit are considered (gradient-based minimization and differential evolution). The main finding is that while for low-INL (linear) ADCs the two methods (LS and ML) give similar results, for practical (almost always nonlinear) ADCs ML is definitely better. Keywords: ADC test, maximum likelihood estimation, least squares method, LS method, four-parameter fit, signal recovery, estimation of signal parameters, gradient-based method, differential evolution. 1. INTRODUCTION Standardized dynamic test methods for analog-to-digital converters (ADCs) ([1]–[3]) are based on comparison of acquired ADC output codes with the ADC input stimulus. The stimulus is not exactly known and cannot be measured with the necessary accuracy, therefore it must be reconstructed from the erroneous ADC output codes acquired during testing. Any inaccuracy in the estimation of ADC stimulus parameters leads to inaccuracy in determination of ADC parameters measured by the dynamic test. The most common way how to recover input signal and estimate its parameters is least squares (LS) fitting. This is also recommended in the standards. According to the theory, LS fitting gives the best estimation under the condition that the 1