Regularized tessellation density estimation with bootstrap aggregation and complexity penalization Matthew Browne School of Health and Human Services, Faculty of Sciences, Engineering and Health, Central Queensland University, QLD, Australia article info Article history: Received 6 October 2009 Received in revised form 18 August 2011 Accepted 2 September 2011 Available online 10 September 2011 Keywords: Regularization Voronoi Tessellation Non-parametric Density estimation Bootstrap aggregation Bagging Information criterion Kernel Model selection abstract Locally adaptive density estimation presents challenges for parametric or non-parametric estimators. Several useful properties of tessellation density estimators (TDEs), such as low bias, scale invariance and sensitivity to local data morphology, make them an attractive alternative to standard kernel techniques. However, simple TDEs are discontinuous and produce highly unstable estimates due to their susceptibility to sampling noise. With the motivation of addressing these concerns, we propose applying TDEs within a bootstrap aggregation algorithm, and incorporating model selection with complexity penalization. We implement complexity reduction of the TDE via sub-sampling, and use information-theoretic criteria for model selection, which leads to an automatic and approximately ideal bias/variance compromise. The procedure yields a stabilized estimator that automatically adapts to the complexity of the generating distribution and the quantity of information at hand, and retains the highly desirable properties of the TDE. Simulation studies presented suggest a high degree of stability and sensitivity can be obtained using this approach. & 2011 Elsevier Ltd. All rights reserved. 1. Introduction Estimating a probability density from a finite sample is a well established in statistics and machine learning [1,21,17]. Though density estimation is an old problem, it remains interesting because of the lack of a complete theory on how to reconcile the competing priorities of data-fidelity/bias and smoothness/ variance in the estimator. Whilst parametric methods approach the issue by applying an explicit prior model to the data, non- parametric approaches make no assumptions about the type of distribution from which the samples are drawn. A comprehensive treatment of non-parametric density estimation is provided by Silverman [2]. Traditionally, non-parametric estimates of a point intensity have relied on kernel smoothing functions [3]. Despite being proposed some time ago [4], it is only recently that the idea of using a geometric tessellations to elucidate properties of point processes has begun to receive significant attention [5]. Okabe [6] provides a review of the use of Voronoi diagrams and Delaunay tessellations for the study of point patterns but without specific applicability to density estimation. Hearne [7,4] completed some early theoretical investigation into the properties of random Delaunay- and Voronoi- based density estimations. Geometric methods have been applied to the detection of clustering struc- ture in data [8] and play a role in novel clustering algorithms [9]. Miller [10] proposed the use of the Voronoi distribution for directly estimating the entropy of a density, but not for approx- imating the inhomogeneous density itself. Binning methods with some similarity to the Voronoi approach have been proposed for astronomical data [11], in which the volume surrounding each point is calculated using a binary tree, the density calculated, and then smoothed using an adaptive kernel. Fluid dynamics applica- tions require a proper estimate of the local density while preser- ving fine density structure and in a comparative study [12],a strong case was made for the use of geometrically adaptive density estimates over kernel smoothing methods. A simple implementation of a tessellation density estimator (TDE) yields estimates that are discontinuous and highly variable (discussed in detail in the following sections). We have previously proposed a method that stabilises the variance using a penalised centroidal Voronoi tessellation (PCVT) [13]. The PCVT smoothes a simple area-based estimate by adapting the arrangement of points themselves through a self-organising process. A disadvantage of PCVT is that, as a computational technique, it does not naturally provide estimates of uncertainty around the density estimates. Barr and Schoenberg [5] compared centroidal Voronoi tessella- tions with non-regularised standard tessellation estimators, and Contents lists available at SciVerse ScienceDirect journal homepage: www.elsevier.com/locate/pr Pattern Recognition 0031-3203/$ - see front matter & 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2011.09.003 E-mail address: m.browne@cqu.edu.au Pattern Recognition 45 (2012) 1531–1539