On Video Textures Generation: A Comparison Between Different Dimensionality Reduction Techniques Wentao Fan and Nizar Bouguila Institute for Information Systems Engineering University of Concordia Montreal, Canada wenta fa@ciise.concordia.ca, bouguila@ciise.concordia.ca Abstract—Video texture is a new type of medium which can provide a new video with a continuously varying stream of images from a recorded video. It is created by reordering the input video frames in a way which can be played without any visual discontinuity. Recently, a new method of generating video textures has been proposed. It first apply principal components analysis (PCA) to extract signatures or patterns from the original video sequence, and then implement an autoregressive process (AR) model to synthesize new video textures. In this paper, we extend this video texture generation method by comparing PCA with other dimensionality reduction techniques such as probabilistic principal components analysis, kernel principal components anal- ysis, independent component analysis, local linear embedding and Isomap. According to our experiments, these approaches prevail the original approach by providing us video textures with better quality. Index Terms—Video texture, computer vision, dimensionality reduction, autoregressive process I. I NTRODUCTION Video texture which has been introduced by Sch¨ odl et al. [1], is a new type of medium that can generate a continuous, infinitely changing stream of images from a recorded video. The term “video texture” is used because it is very similar to image textures. This technique can be considered as video- based rendering (VBR) because it has similar features with image-based rendering (IBR) technique [2], that is, both of them are able to reuse the already existing resources to synthe- size new objects. For video textures, a recorded video is used to make a new video stream without any visual discontinuity by changing the order of the original frames. This would be useful in movie and game industries, since it may create new objects by reusing existing resources so that time and human resources may be saved. More applications may be found in [3] [4] [5] [6]. However, same as the original video textures technique, all of these works can only generate new video by just switching the order of frames and the result would suffer from ‘dead-ends’. Recently in computer vision, there are increased number of researches on time series analysis to model the dynamical characteristics of complex systems. Autoregressive (AR) pro- cess [7] is a tool used for understanding and predicting future values in a time series. In [8], Fitzgibbon have introduced a new method for creating video textures by applying principal components analysis (PCA) and AR process. All frames in the generated video are new and consist with the motions in the original video, and ‘dead ends’ would never appear. In [9], Campbell et al. have extended this approach to work with strongly non-linear sequences by applying a spline and a combined appearance model. The rest of this paper is organized as follows: We briefly introduce different dimensionality reduction techniques in sec- tion 2. Then, we compare the experimental results of applying different dimensionality reduction techniques to generate video textures in section 3. Finally, in section 4 we conclude and describe some topics for future work. II. DIMENSIONALITY REDUCTION APPROACHES Dimensionality reduction is an important research topic in the area of data analysis. The goal of dimensionality reduction techniques is to discover a low-dimensional subspace that best represents a given set of data points. In this paper, we extend the work of Fitzgibbon [8] by comparing PCA with other representative dimensionality reduction techniques to extract signatures from video frames and then synthesize new video textures. The techniques we have applied are: probabilistic principal components analysis, kernel principal components analysis, Isomap, local linear embedding and independent component analysis. In this section, we will give a brief introduction for each technique individually. A. Probabilistic Principal Component Analysis In [10], Bishop has proposed a probabilistic model for PCA by showing that PCA can be represented as the maximum likelihood solution of a probabilistic latent variable model. This novel form of PCA is known as probabilistic principal component analysis (PPCA). PPCA can be formulated by first choosing an explicit M - dimensional latent variable z corresponding to the principal component subspace and then sampling the D-dimensional observed variable x conditioned on this latent variable. We define a Gaussian prior distribution p(z) for the latent variable, and a conditional Gaussian distribution p(x|z) for the observed Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics San Antonio, TX, USA - October 2009 978-1-4244-2794-9/09/$25.00 ©2009 IEEE 5284