Three-dimensional Gaussian Mixture Texture Model Michal Haindl and Vojtˇ ech Havl´ ıˇ cek Institute of Information Theory and Automation of the CAS Prague, Czech Republic 182 08 Email: {haindl,havlicek}@utia.cas.cz Abstract—Visual texture modeling based on multidimensional mathematical models is the prerequisite for both robust material recognition as well as for image restoration, compression or numerous physically correct virtual reality applications. A novel multispectral visual texture modeling method based on a de- scriptive, unusually complex, three-dimensional, spatial Gaussian mixture model is presented. Texture synthesis benefits from easy computation of arbitrary conditional distributions from the model. The model is inherently multispectral thus it does not suffer with the spectral quality compromises of the spectrally factorized alternative approaches. The model is especially well suited for multispectral textile textures and it can also describe the most advanced textural representation in the form of a bidirectional texture function (BTF). I. I NTRODUCTION Human observer’s visual scene recognition is based on shapes and materials. Unfortunately, the surface material ap- pearance vastly changes under variable observation conditions which significantly complicates and negatively affects its mathematical synthesis as well as machine analysis. Reliable computer-based interpretation of visual information which would approach human cognitive capabilities is very challeng- ing and impossible without significant improvement of the cor- responding sophisticated visual information models capable to handle huge variations of possible observation conditions. The appropriate paradigm for such a surface material reflectance function models is a multidimensional visual texture. Texture synthesis approaches may be divided primarily into sampling and model-based methods. Sampling methods [1], [2], [3], [4], [5], [6] rely on sophisticated sampling from real texture measurements while the model-based techniques [7], [8], [9], [10], [11], [12], [13] describe texture data using multidimensional mathematical models and their synthesis is based on the estimated model parameters only. Generative visual texture models are useful not only for modelling physically correct virtual objects material surfaces in virtual or augmented reality environments, image restoration or compression but also for contextual recognition applications such as segmentation, classification or image retrieval. Physically correct surface material reflectance model (RM) is sixteen-dimensional function [14] RM (λ i ,x i ,y i ,z i ,t i ,θ i ,ψ i ,λ v ,x v ,y v ,z v ,t v ,θ v ,ψ v ,θ i,T ,θ v,T ). RM describes incident light with spectral value λ i illuminating surface location x i ,y i ,z i in time t i under spherical reflectance angle θ i ,ψ i and observed at time t v from surface location x v ,y v ,z v under spherical reflectance angle θ v ,ψ v and spec- trum λ v . θ i,T ,θ v,T are the corresponding transmittance angles. The model height parameters z i ,z v indicate that even radiance along light rays is not constant but depends on the height. Such a RM model is too complex and there neither exist any measurement of such data nor any mathematical representation allowing its synthesis. One of the early compromised attempts to capture real material appearance was done by Nicodemus et al. [15] and later elaborated by Dana et al. [16] in the form of Bidirectional Texture Function (BTF). Even if a BTF model assumes several strong simplifying assumptions [17], [18], [14] its measurement, compression and synthesis is on the leading edge of current mathematical modelling and techno- logical capabilities. BTF is a seven-dimensional function [14] which considers not only measurement dependency on planar material position and spectral channel but also its dependence on illumination and viewing angles: BTF θi,φi,θv,φv (˜ r) (1) where θ,φ are elevation and azimuthal angles of illumination and view direction vector, the multiindex ˜ r = [r 1 ,r 2 ,r 3 ] specifies planar horizontal and vertical position in a material sample image and r 3 is the spectral index. Reliable parameters estimation of such a seven-dimensional stochastic model is very difficult not only because it requires very demanding numerical optimization but because the learning textural data are always too limited to obtain robust and reliable estimates. The solution is to factorize the original seven-dimensional measurement space into a set of less dimensional textural fac- tors. The realistic modeling strives not necessarily to recover the exact pixel-wise correspondence with some original target texture but rather a texture which is visually indiscernible from the original one. In our previous paper [19] we have introduced three two- dimensional probabilistic mixture models, where a measured 3D multi-spectral texture had to be spectrally factorized and the corresponding multivariate mixture models were further learned from single orthogonal mono-spectral components and used to synthesise and enlarge these mono-spectral factor components. The presented model (BTF-3DGMM), on the contrary, is fully multispectral and thus it does not need to compromise spectral modeling quality in multicoloured textures. We applied this model for simpler task of high quality texture restoration [20] where the model can exploit information from corrupted textural data. The presented ap- plication to BTF (color / multispectral) texture synthesis is 2016 23rd International Conference on Pattern Recognition (ICPR) Cancún Center, Cancún, México, December 4-8, 2016 978-1-5090-4846-5/16/$31.00 ©2016 IEEE 2026