Learning High-order Generative Texture Models Ralph Versteegen, Georgy Gimel’farb and Patricia Riddle Department of Computer Science The University of Auckland Auckland 1142, New Zealand rver017@aucklanduni.ac.nz, {g.gimelfarb, p.riddle}@auckland.ac.nz ABSTRACT We introduce a new simple framework for texture modelling with Markov–Gibbs random fields (MGRF). The framework learns texture-specific high order pixel interactions described by feature functions of signal patterns. Currently, modelling of high order interactions is almost exclusively achieved by linear filtering. Instead we investigate ‘binary pattern’ (BP) features which are faster to compute and describe quite dif- ferent properties than linear filters. The features are similar to local binary patterns (LBPs) — previously not applied as MGRF features — but with learnt shapes. In contrast to the majority of MGRF models the set of features used is learnt from the training data and is heterogeneous. This paper shows how these features can be efficiently selected by nesting the models. Each new layer corrects errors of the previous model while allowing incremental composition of the features, and uses validation data to decide the stop- ping point. The framework also reduces overfitting and speeds learning due to a feasible number of free parame- ters to be learnt at each step. Texture synthesis results of the proposed texture models were quantitatively evaluated by a panel of observers, showing higher order BP features resulted in significant improvements on regularly and irreg- ularly structured textures. Categories and Subject Descriptors I.2 [Artificial Intelligence]; I.2.10 [Vision and Scene Understanding]: Texture; I.4 [Image Processing and Computer Vision]; I.4.7 [Feature Measurement]: Tex- ture General Terms Theory; algorithms; experimentation 1. INTRODUCTION Texture modelling is important to many computer vi- sion and image processing tasks such as image segmenta- Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- tion on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- publish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. IVCNZ ’14, November 19 - 21 2014, Hamilton, New Zealand Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-3184-5/14/11$15.00 http://dx.doi.org/10.1145/2683405.2683420. tion, inpainting, classification, synthesis, anomaly (defect) detection, and so forth. Although successful specialised al- gorithms for solving these problems have been developed, generative probabilistic models which provide explicit im- age probabilities for a specific texture class remain appeal- ing. These models may be applied not only to all of the above tasks, but also to others where appearance priors or feature extraction are needed, and they are of interest to understanding human vision. Unlike discriminative models, generative models need to capture texture features that are significant to human perception in order to be successful. The most prevalent tool for image and texture modelling are Markovian undirected graphical models, a.k.a. Markov random fields (MRFs). A Markov–Gibbs random field (MGRF) is an MRF specified with an explicit Gibbs prob- ability distribution (GPD). A GPD is factorised over com- plete subgraphs (cliques) of an interaction graph with vari- ables/pixels as nodes. which Factors quantify the strength of interactions between those variables and can be described in terms of feature functions which identify certain signal configurations (patterns), each of which has a correspond- ing weight/parameter. The order of a feature is its arity, and in general higher order one cannot be decomposed into lower order features. MGRFs and other statistical texture mod- els reduce images to a vector of statistics of image features, which are assumed sufficient to describe the texture. His- torically MRF models used statistics of pairs of pixels [3, 5]. However higher-order MGRFs have become more common as they are recognised to be necessary for more expressive models of natural images and textures, by abstracting be- yond pixels [7, 18, 25]. In addition, since regularly tiled tex- tures have strong long range correlations between nearby tiles it is natural to learn an interaction structure specific to the texture. Yet it is still almost unheard of in computer vision and image modelling for higher-order MRF structure to be learnt rather than hand selected. The higher-order MGRFs in use nearly exclusively ap- ply linear filters as feature functions, with statistics of the filter responses, such as means and covariances [16] or his- tograms [25], forming a description vector. More general high order features require specification of clique shapes (e.g. circular rings for local binary patterns (LBPs) [14, 15]) and a reasonably parametrised functional form from amongst a combinatorially huge space. In order to explore this ne- glected possibility, this paper describes a model nesting pro- cedure which greedily adds features to a model to correct statistical differences between the training data and current model, and attempts to build higher order features by com-