J Geod (2012) 86:123–136 DOI 10.1007/s00190-011-0501-z ORIGINAL ARTICLE Fast integer least-squares estimation for GNSS high-dimensional ambiguity resolution using lattice theory S. Jazaeri · A. R. Amiri-Simkooei · M. A. Sharifi Received: 14 December 2010 / Accepted: 15 July 2011 / Published online: 30 July 2011 © Springer-Verlag 2011 Abstract GNSS ambiguity resolution is the key issue in the high-precision relative geodetic positioning and navigation applications. It is a problem of integer programming plus integer quality evaluation. Different integer search estima- tion methods have been proposed for the integer solution of ambiguity resolution. Slow rate of convergence is the main obstacle to the existing methods where tens of ambiguities are involved. Herein, integer search estimation for the GNSS ambiguity resolution based on the lattice theory is proposed. It is mathematically shown that the closest lattice point prob- lem is the same as the integer least-squares (ILS) estimation problem and that the lattice reduction speeds up searching process. We have implemented three integer search strate- gies: Agrell, Eriksson, Vardy, Zeger (AEVZ), modification of Schnorr–Euchner enumeration (M-SE) and modification of Viterbo-Boutros enumeration (M-VB). The methods have been numerically implemented in several simulated exam- ples under different scenarios and over 100 independent runs. The decorrelation process (or unimodular transforma- tions) has been first used to transform the original ILS prob- lem to a new one in all simulations. We have then applied Electronic supplementary material The online version of this article (doi:10.1007/s00190-011-0501-z) contains supplementary material, which is available to authorized users. S. Jazaeri (B ) · M. A. Sharifi Department of Surveying and Geomatics Engineering, College of Engineering, University of Tehran, Tehran, Iran e-mail: jazayeri@ut.ac.ir M. A. Sharifi e-mail: sharifi@ut.ac.ir A. R. Amiri-Simkooei Department of Surveying Engineering, Faculty of Engineering, University of Isfahan, 81746-73441 Isfahan, Iran e-mail: ar_amiri@yahoo.com different search algorithms to the transformed ILS prob- lem. The numerical simulations have shown that AEVZ, M-SE, and M-VB are about 320, 120 and 50 times faster than LAMBDA, respectively, for a search space of dimen- sion 40. This number could change to about 350, 160 and 60 for dimension 45. The AEVZ is shown to be faster than MLAMBDA by a factor of 5. Similar conclusions could be made using the application of the proposed algorithms to the real GPS data. Keywords Integer least-squares estimation · GNSS ambiguity resolution · Lattice theory · Pohst enumeration · Schnorr–Euchner enumeration 1 Introduction High-precision GNSS positioning is achieved using the car- rier phase observables in the relative positioning mode. GNSS relative positioning is used for many high-precision applications such as surveying, mapping, GIS, and precise navigation. A prerequisite to this is the successful determina- tion of the integer double difference carrier phase ambiguity parameters. Mathematically, double difference carrier phase observation equation is a mixed integer nonlinear model. Lin- earizing the carrier phase observation equation yields the fol- lowing mixed integer linear model (Teunissen 1995; Xu et al. 1995; Xu 2006): y = Aa + Bb + e (1) where y is a t -dimensional vector of observed minus approx- imate double difference carrier phase observations, a is an n-dimensional integer vector, b is an m-dimensional real-val- ued vector, e is the error vector of observations, and A and B are the t × n and t × m real-valued matrices, respectively. 123