Finite-Length Markov Processes with Constraints Franc ¸ois Pachet, Pierre Roy, Gabriele Barbieri Sony CSL Paris {pachet,roy,barbieri}@csl.sony.fr Abstract Many systems use Markov models to generate finite-length sequences that imitate a given style. These systems often need to enforce specific con- trol constraints on the sequences to generate. Un- fortunately, control constraints are not compatible with Markov models, as they induce long-range de- pendencies that violate the Markov hypothesis of limited memory. Attempts to solve this issue us- ing heuristic search do not give any guarantee on the nature and probability of the sequences gener- ated. We propose a novel and efficient approach to controlled Markov generation for a specific class of control constraints that 1) guarantees that generated sequences satisfy control constraints and 2) fol- low the statistical distribution of the initial Markov model. Revisiting Markov generation in the frame- work of constraint satisfaction, we show how con- straints can be compiled into a non-homogeneous Markov model, using arc-consistency techniques and renormalization. We illustrate the approach on a melody generation problem and sketch some real- time applications in which control constraints are given by gesture controllers. 1 Introduction Markov processes are a popular modeling tool used in con- tent generation applications, such as text generation, music composition and interaction. Markov processes are based on the “Markov hypothesis” which states that the future state of a sequence depends only on the last state, i.e., p(s i |s 1 ,...,s i−1 )= p(s i |s i−1 ). The Markovian aspects of musical sequences have long been acknowledged, see e.g. [Brooks et al., 1992]. Many attempts to model musical style have therefore exploited Markov chains in various ways [Nierhaus, 2009], notably se- quence generation. In practice, Markov models are often estimated by count- ing occurrences and transitions in a corpus of training se- quences. Once the model is learn, sequences can be generated simply by random walk: the first item is chosen randomly us- ing the prior probabilities; then, a continuation is drawn using the model, and appended to the first item. This is iterated to produce a sequence of length L. This process has the advan- tage of being simple to implement and efficient. For instance, the Continuator [Pachet, 2002] uses a Markov model to react interactively to music input. Its success was largely due to its capacity to faithfully imitate arbitrary musi- cal styles, at least for relatively short time frames. Indeed, the Markov hypothesis basically holds for most melodies played by users (from children to professionals) in many styles of tonal music (classical, jazz, pop, etc.). The other reason of its success is the variety of outputs produced for a given in- put. All continuations produced are stylistically convincing, thereby giving the sense that the system creates infinite, but plausible, possibilities from the user’s style. With the Continuator, a user typically plays a musical phrase using a MIDI keyboard. The phrase is then converted into a sequence of symbols, representing a given dimension of music, such as pitch, duration, or velocity. The sequence is then analyzed by the system to update the Markov model. When the phrase is finished, typically after a certain tempo- ral threshold has passed, the system generates a new phrase using the model built so far. The user can then play another phrase, or interrupt the phrase being played, depending on the chosen interaction mode. It was shown that such incremental learning creates engaging dialogs with users, both with pro- fessional musicians and children [Addessi and Pachet, 2005]. Other systems such as Omax [Cont et al., 2007] followed the same principle with similar results. In such interactive contexts, the control problem manifests itself under different forms: • The so-called zero-frequency problem arises during ran- dom walk, when an item with no continuation is cho- sen (see, e.g. [Chordia et al., 2010]). Many strategies have been devised to circumvent this problem, including restarting the walk from scratch [Dubnov et al., 2003]. • The end point or drift problems [Davismoon and Eccles, 2010] concern the fact that the generated sequence can violate musical constraints holding, e.g., on the pitch range of the melody. • User control constraints. In a musical context, the user may want the sequence to be globally ascending, pitch- wise, or to follow an arbitrary pitch contour. These con- straints can be a consequence of a particular gesture, de- 635 Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence