Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Ron Rubinstein ∗ , Michael Zibulevsky ∗ and Michael Elad ∗ Abstract The K-SVD algorithm is a highly effective method of training overcomplete dic- tionaries for sparse signal representation. In this report we discuss an efficient im- plementation of this algorithm, which both accelerates it and reduces its memory consumption. The two basic components of our implementation are the replacement of the exact SVD computation with a much quicker approximation, and the use of the Batch-OMP method for performing the sparse-coding operations. Batch-OMP, which we also present in this report, is an implementation of the Orthogonal Matching Pursuit (OMP) algorithm which is specifically optimized for sparse-coding large sets of signals over the same dictionary. The Batch-OMP imple- mentation is useful for a variety of sparsity-based techniques which involve coding large numbers of signals. In the report, we discuss the Batch-OMP and K-SVD implementations and analyze their complexities. The report is accompanied by Matlab  toolboxes which implement these techniques, and can be downloaded at http://www.cs.technion.ac.il/ ~ ronrubin/software.html. 1 Introduction Sparsity in overcomplete dictionaries is the basis for a wide variety of highly effective signal and image processing techniques. The basic model suggests that natural signals can be efficiently explained as linear combinations of prespecified atom signals, where the linear coefficients are sparse (most of them zero). Formally, if x is a column signal and D is the dictionary (whose columns are the atom signals), the sparsity assumption can be described by the following sparse approximation problem, ˆ γ = Argmin γ ‖γ ‖ 0 Subject To ‖x − Dγ ‖ 2 2 ≤ ǫ. (1.1) In this formulation, γ is the sparse representation of x , ǫ the error tolerance, and ‖·‖ 0 is the ℓ 0 pseudo-norm which counts the non-zero entries. The sparse approximation problem, which is known to be NP-hard, can be efficiently solved using several available * Computer Science Department, Technion – Israel Institute of Technology, Haifa 32000 Israel. 1 Technion - Computer Science Department - Technical Report CS-2008-08.revised - 2008