Addressing NP-Complete Puzzles with Monte-Carlo Methods 1 Maarten P.D. Schadd and Mark H.M. Winands H. Jaap van den Herik and Huib Aldewereld 2 Abstract. NP-complete problems are a challenging task for researchers, who investigate tractable versions and attempt to generalise the meth- ods used for solving them. Over the years a large set of successful standard methods have been developed. We mention A* and IDA* which have proven to be reasonably successful in solving a set of NP-complete problems, particularly single-agent games (puzzles). However, sometimes these methods do not work well. The intrigu- ing question then is whether there are new methods that will help us out. In this paper we investigate whether Monte-Carlo Tree-Search (MCTS) is an interesting alternative. We propose a new MCTS vari- ant, called Single-Player Monte-Carlo Tree-Search (SP-MCTS). Our domain of research is the puzzle SameGame. It turned out that our SP-MCTS program gained the highest scores so far on the standard- ised test set. So, SP-MCTS can be considered as a new method for addressing NP-complete puzzles successfully. 1 INTRODUCTION Creating and improving solvers for tractable versions of NP- complete problems is a challenging task in the field of Artificial In- telligence research. As Cook [9] proved: all problems in the class of NP-complete problems are translatable to one another [16]. This im- plies that a solution procedure for one problem also holds for other problems. Otherwise stated: if an effective method is found to solve a particular instance of a problem, many other problems may be solved as well using the same method. Games are often NP-complete problems. The rules for games are well-defined and it is easy to compare different methods. For our in- vestigations we have chosen a one-person perfect-information game (a puzzle 3 ) called SameGame. In Section 2 we will prove that this puzzle is NP-complete. The traditional methods for solving puzzles, such as the 15×15 puzzle and Sokoban, are A* [15] or IDA* [19]. Other problems, such as the Travelling Salesman Problem (TSP) [3] require different meth- ods (e.g., Simulated Annealing [12] or Neural Networks [23]). These methods have been shown to solve the puzzles mentioned above rea- sonably well. An example of a practical and successful use of these methods are pathfinders which are, for example, used inside an in- creasing number of cars. A drawback of the methods is that they need 1 This contribution is a revised version of an article under submission to CG 2008. 2 Maastricht University, Maastricht, The Netherlands, email: {maarten.schadd, m.winands, herik, h.aldewereld}@micc.unimaas.nl 3 Although arbitrary, we will call these one-player games with perfect infor- mation for the sake of brevity puzzles. an admissible heuristic evaluation function. The construction of such a function may be difficult. An alternative to these methods can be found in Monte-Carlo Tree Search (MCTS) [7, 10, 18] because it does not need an admissible heuristic. Especially in the game of Go, which has a large search space [5], MCTS methods have proven to be successful [7, 10]. In this paper we will investigate how MCTS addresses NP-complete puzzles. For this purpose, we introduce a new MCTS variant called SP-MCTS. The course of the paper is as follows. In Section 2 we present the background and rules of SameGame. Also, we prove that SameGame is NP-complete. In Section 3 we discuss why classical methods are not suitable for SameGame. Then we introduce our SP-MCTS ap- proach in Section 4. Experiments and results are given in Section 5. Section 6 shows our conclusions and indicates future research. 2 SAMEGAME We start by presenting some background information on SameGame in Subsection 2.1. Subsequently we explain the rules in Subsection 2.2. Finally, we prove that SameGame is NP-complete in Subsection 2.3. 2.1 Background SameGame is a puzzle invented by Kuniaki Moribe under the name Chain Shot! in 1985. It was distributed for Fujitsu FM-8/7 series in a monthly personal computer magazine called Gekkan ASCII [20]. The puzzle was afterwards re-created by Eiji Fukumoto under the name of SameGame in 1992. So far, the best program for SameGame has been developed by Billings [24]. 2.2 Rules SameGame is played on a rectangular vertically-placed 15×15 board initially filled with blocks of 5 colours at random. A move consists of removing a group of (at least two) orthogonally adjacent blocks of the same colour. The blocks on top of the removed group will fall down. As soon as empty columns occur, the columns to the right are shifted to the left. For each removed group points are rewarded. The amount of points is dependent on the number of blocks removed and can be computed by the formula (n - 2) 2 , where n is the size of the removed group. We show two example moves in Figure 1. When the ‘B’ group in the third column of position 1(a) is played, it will be removed from the game and the ‘C’ block on top will fall down, resulting in position