IEEE TRANSACTIONS ON PATTFRN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-6, NO. 1, JANUARY 1984 sor in the Department of Computer Science, Wayne State University, Detroit, MI. In 1974, he joined the Department of Computer Science, Michigan State University, where he is currently a Professor. He served as the Program Director of the Intelligent Systems Program at the Na- tional Science Foundation from September 1980 to August 1981. His research interests are in the areas of pattern recognition and image processing. Dr. Jain is a member of the Association for Computing Machinery, the Pattern Recognition Society, and Sigma Xi. He is also an Advisory Editor of Pattern Recognition Letters. K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality SHOKRI Z. SELIM AND M. A. ISMAIL, MEMBER, IEEE Abstract-The K-means algorithm is a commonly used technique in cluster analysis. In this paper, several questions about the algorithm are addressed. The clustering problem is first cast as a nonconvex mathe- matical program. Then, a rigorous proof of the finite convergence of the K-means-type algorithm is given for any metric. It is shown that under certain conditions the algorithm may fail to converge to a local minimum, and that it converges under differentiability conditions to a Kuhn-Tucker point. Finally, a method for obtaining a local-mini- mum solution is given. Index Terms-Basic ISODATA, cluster analysis, K-means algorithm, K-means convergence, numerical taxonomy. I. INTRODUCTION -MEANS-type algorithms for exploratory data clustering and analysis are very popular and well known [1] -[8]. The main idea behind these techniques is the minimization of a certain criterion function usually taken up as a function of the deviations between all patterns from their respective cluster centers. Usually, the minimization of such a criterion function is sought utilizing an iterative scheme which starts with an arbi- trary chosen initial cluster configuration of the data, then alters the cluster membership in an iterative manner to obtain a better configuration. The sum of squared Euclidean distances criterion has been adopted in most of the studies related to these algorithms, due to its computational simplicity, since the cluster at each iteration can be calculated in a straightfor- ward manner. A K-means algorithm alternates between two major steps until a stopping criterion is satisfied. These steps Manuscript received July 12, 1982; revised January 24, 1983. This work was supported by the University of Petroleum and Minerals, Dhahran, Saudi Arabia. S. Z. Selim is with the Department of Systems Engineering, Univer- sity of Petroleum and Minerals, Dhahran, Saudi Arabia. M. A. Ismail is with the Department of Computer Science, University of Windsor, Windsor, Canada. are mainly the distribution of patterns among clusters utiliz- ing a specific classifier (usually the minimum Euclidean dis- tance classifier: MEDC), and the updating of cluster centers [9] -[11 ]. Incorporation of some heuristic procedures into the above- mentioned iterative scheme results in the well-known ISODATA algorithm of Ball and Hall [12], which may be considered as another sophisticated form of the original K-means. The most important heuristic procedures in ISODATA are those allow- ing cluster lumping and cluster splitting. Several extensive studies dealing with comparative analysis of different clustering methods have been conducted recently, utilizing both simulated data sets, e.g., [2], [13], and practi- cal data, e.g., [6], [14]-[16]. These studies recommend the K-means algorithm as one of the best clustering methods avail- able. Several evaluation criteria were adopted in such studies and a variety of techniques were compared simultaneously. Moreover, fuzzy versions of the K-means algorithm have been reported in a series of papers by Ruspini [17], [18] Dunn [19], and Bezdek [19]-[22], where each pattern is allowed to have membership functions to all clusters rather than having a distinct membership to exactly one cluster. The usefulness of the K-means-type algorithms is not ques- tionable, and the extensive experimentation with these algo- rithms using practical data suggests and establishes the appli- cability and practicality of such techniques. Although it is found that such algorithms converge when applied to different data sets from a wide range of applications, no rigorous theo- rem for the convergence of the K-means-type algorithms exists to date, and the question of convergence of such methods remain open [1], [3] , and [22] . In this paper, a rigorous proof of convergence of the K-means- type algorithm is given in a generalized form. Moreover, local optimality of solutions obtained has been investigated, where it is shown that under certain conditions, the K-means algo- 0162-8828/84/0100-0081$01.00 © 1984 IEEE 81