Graph Cuts for Minimizing Robust Higher Order Potentials Pushmeet Kohli Microsoft Research Cambridge pkohli@microsoft.com L’ubor Ladick´ y Philip H. S. Torr Oxford Brookes University {lladicky,philiptorr}@brookes.ac.uk Abstract Energy functions defined on higher order cliques can model complex interactions between groups of random vari- ables. They have the capability of modelling the rich statis- tics of natural scenes which can be used for many applica- tions in computer vision. However, these energies are sel- dom used in practice, primarily due to the lack of efficient algorithms for minimizing them. In this paper we introduce a novel family of higher order potentials called the Robust P n model. This family is a generalization of the P n Potts model class of potentials which was recently introduced in computer vision. We show that energy functions contain- ing such potentials can be solved using the expansion and swap move algorithms for approximate energy minimiza- tion. Specifically, we prove that the optimal swap and ex- pansion moves for energy functions composed of these po- tentials can be computed by solving a st-mincut problem. For functions of binary variables, our method is able to pro- duce the globally optimal solution in polynomial time. Fur- ther, it is extremely efficient and can handle energies defined on cliques involving thousand of variables. 1. Introduction Higher order potential functions can model complex in- teractions among sets of random variables. This flexibil- ity enables them to encode sophisticated statistics of nat- ural scenes which cannot be expressed using pairwise po- tentials. Researchers have used higher order potentials to produce excellent results for a number of challenging Com- puter Vision problems such as image restoration [18], de- noising [12], optical flow [19], texture synthesis [16], and segmentation [7]. However, the lack of efficient algorithms for performing inference in these models has limited their popularity. The runtime complexity of commonly used in- ference algorithms such as belief propagation (BP) or Tree- reweighted message passing (TRW)[23] grows exponen- tially with the clique size, which makes them inapplicable to functions defined on even moderate sized cliques [12]. Recent work on message passing algorithms has been partly successful in improving their performance for certain classes of higher order potential functions. Lan et al. [12] proposed approximation methods for BP to make efficient inference possible in higher order MRFs. This was followed by the work of Potetz [17] in which he showed how belief propagation can be efficiently performed in graphical mod- els containing moderately large cliques. However, as these methods are based on BP, they are quite slow and take min- utes or even hours to converge. Kohli et al. [7] recently showed how certain higher order clique potentials can be minimized using the α-expansion and αβ-swap move mak- ing algorithms for approximate energy minimization [2]. They introduced a class of higher order potentials called the P n Potts model and proved that the optimal expansion and swap moves for energy functions containing these po- tentials can be computed in polynomial time by solving a st-mincut problem. In this paper we introduce a new family of higher or- der potentials called the Robust P n model which is a gen- eralization of the P n Potts class. The potential functions belonging to this family are parameterized with a trunca- tion parameter Q which controls their robustness. We will show how energy functions composed of these robust po- tentials can be minimized using move making algorithms such as α-expansion and αβ-swap. Specifically, we show how the optimal swap and expansion moves for such poten- tials can be found by solving an st-mincut problem. In fact, for functions of binary variables, our method produces the globally optimal solution. The complexity of our method increases linearly with the clique size and thus it is able to handle cliques composed of thousands of latent variables. In a related paper on object segmentation, we have shown that the Robust P n model potentials produce more accu- rate segmentation results compared to those obtained using pairwise and/or P n Potts potential functions [8]. Outline of the Paper A brief outline of the paper follows. In section 2 we discuss the energy minimization problem and describe the graph cut based expansion and swap move algorithms. In section 3 we introduce the Robust P n model and discuss its relationship to the P n Potts model. We also describe how it can be used to approximate any con- cave increasing consistency potential function. In section 1