arXiv:0910.3482v1 [math.NT] 19 Oct 2009 RATIONAL APPROXIMATION OF THE MAXIMAL COMMUTATIVE SUBGROUPS OF GL(n, R) OLEG N. KARPENKOV 1) , ANATOLY M. VERSHIK 2) Abstract. How to find “best rational approximations” of maximal commutative sub- groups of GL(n, R)? In this paper we pose and make first steps in the study of this problem. It contains both classical problems of Diophantine and simultaneous approxi- mations as a particular subcases but in general is much wider. We prove estimates for n = 2 for both totaly real and complex cases and write the algorithm to construct best approximations of a fixed size. In addition we introduce a relation between best approxi- mations and sails of cones and interpret the result for totally real subgroups in geometric terms of sails. Contents Introduction: the problem and its relationships 2 1. Rational approximations of MCRF-groups 8 1.1. Regular subgroups and Markoff-Davenport forms 8 1.2. Rational subgroups and their sizes 9 1.3. Discrepancy functional and approximation model 10 2. Diophantine approximations and MCRS-group approximations 10 3. General approximations in two-dimensional case 11 3.1. Hyperbolic case 11 3.2. Non-hyperbolic case 17 4. Simultaneous approximations in R 3 and MCRS-group approximations 18 4.1. General construction 19 4.2. A ray of non-hyperbolic operator 19 4.3. Two-dimensional golden ratio 19 References 20 Date : 14 October 2009. Key words and phrases. Maximal commutative subgroups, centralizers, Diophantine approximations, Markoff-Davenport forms, sail of simplicial cones. MSC2010: 11J13, 11K60, 11J70. 1) Supported by RFBR SS-709.2008.1 and FWF No. S09209. 2) Supported by NSh-2460.2008.1, RFBR 08-01-00379, and RFBR 09-01-12175 OFI-M. 1