Noname manuscript No. (will be inserted by the editor) A Generative Model and a Generalized Trust Region Newton Method for Noise Reduction Seppo Pulkkinen · Marko M. M¨ akel¨ a · Napsu Karmitsa Received: date / Accepted: date Abstract In practical applications related to, for instance, machine learning, data mining and pattern recognition, one is commonly dealing with noisy data lying near some low- dimensional manifold. A well-established tool for extracting the intrinsically low-dimensional structure from such data is principal component analysis (PCA). Due to the inherent limita- tions of this linear method, its extensions to extraction of nonlinear structures have attracted increasing research interest in recent years. Assuming a generative model for noisy data, we develop a probabilistic approach for separating the data-generating nonlinear functions from noise. We demonstrate that ridges of the marginal density induced by the model are viable estimators for the generating functions. For projecting a given point onto a ridge of its estimated marginal density, we develop a generalized trust region Newton method and prove its convergence to a ridge point. Accuracy of the model and computational efficiency of the projection method are assessed via numerical experiments where we utilize Gaussian kernels for nonparametric estimation of the underlying densities of the test datasets. Keywords principal manifold · noise reduction · generative model · ridge · density estimation · trust region · Newton method 1 Introduction Machine learning, data mining and pattern recognition are typical tasks, where one is com- monly dealing with noisy data lying near some low-dimensional manifold. Extraction of the intrinsically low-dimensional structure from such data is an essential task in many applica- tions. The usual tool for this purpose is principal component analysis (PCA) that can be used S. Pulkkinen (B) Turku Centre for Computer Science (TUCS) and University of Turku, 20014 Turku, Finland E-mail: seppo.pulkkinen@utu.fi M.M. M¨ akel¨ a University of Turku, 20014 Turku, Finland E-mail: makela@utu.fi N. Karmitsa University of Turku, 20014 Turku, Finland E-mail: napsu@karmitsa.fi