Learning and Inference for General Non Submodular Pairwise MRFs Vibhav Vineet Jonathan Warrell Philip Torr Oxford Brookes University {vibhav.vineet-2010, jwarrell, philiptorr}@brookes.ac.uk 1. Tiered Move Making Algorithm A large number of problems in computer vision can be modeled as energy minimization problems in a markov ran- dom field (MRF) framework. Many methods have been de- veloped over the years for efficient inference, especially in pairwise MRFs. In general there is a trade-off between the complexity/efficiency of the algorithm and its convergence properties, with certain problems requiring more complex inference to handle general pairwise potentials. Graph- cuts based α-expansion performs well on certain classes of energies, and sequential tree reweighted message passing (TRWS) and loopy belief propagation (LBP) can be used for non-submodular cases. These methods though suffer from poor convergence and often oscillate between solutions. In this paper, we propose a tiered move making algorithm which is an iterative method. Each move to the next config- uration is based on the current labeling and an optimal tiered move, where each tiered move requires one application of the dynamic programming based tiered labeling method in- troduced in Felzenszwalb et. al. [1]. The algorithm con- verges to a local minimum for any general pairwise poten- tial, and we give a theoretical analysis of the properties of the algorithm, characterizing the situations in which we can expect good performance. 2. Experiments We demonstrate effectiveness of our move making al- gorithm (t-move) on many MRF problems such as stereo, segmentation, photomontage, and denoising. In order to show the generalization ability of the t-move algorithm in handling any pairwise potential, we compare the minimum energy values achieved by different benchmark methods on random energy functions. We compare against a range of other approximate algorithms, including α-expansion, loopy belief propagation (LBP), quadratic pseudo-boolean optimization (QPBO) and sequential tree reweighted mes- sage passing (TRWS) on various benchmark images. On stereo images, our algorithm is able to achieve al- most 1% to 2% lower energies than α-expansion, QPBO and LBP methods, though TRWS achieves slightly better energies than our method. Figure 1 shows input and out- put of our algorithm on the tsukuba image with 16 dispar- Figure 1. Our tiered move making algorithm achieves a minimum energy value 0.3% lower than that achieved by α-expansion on truncated L1-norm pairwise potential for stereo on the tsukuba im- age from [2] with 16 disparity labels. ity labels. For binary segmentation problems, we achieve the global minimum energy values on all images, as does α-expansion. Further, we achieve almost 2%-3% lower en- ergy values than the original tiered labeling method [1] on all segmentation images. We achieve almost 0.2%-0.5% lower energy than TRWS and α-expansion on a photomon- tage problem and almost 5% lower than α-expansion on a denoising problem although our energies are slightly higher than TRWS. We show our algorithm consistently to achieve lower energies than all methods except for TRWS, with which we remain competitive. On a random energy function with random unary and pairwise potentials, we observe that the QPBO method consistently does quite well, followed by our tiered move method. We achieve almost 10% less en- ergy value than TRWS on the random energies. We also test our approach on object class segmentation on pascalVOC- 10 segmentation dataset. We learn a general pairwise po- tential, and use our t-move method for inference. References [1] P. F. Felzenszwalb and O. Veksler. Tiered scene labeling with dynamic programming. In CVPR, pages 3097–3104, 2010. 1 [2] R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother. A comparative study of energy minimization methods for markov random fields with smoothness-based priors. PAMI, 30(6):1068–1080, 2008. 1 1