Motion Pattern Analysis for Modeling and Recognition of Complex Human Activities Nandita M. Nayak, Ricky J.Sethi, Bi Song, Amit K. Roy-Chowdhury 1 Abstract Activity recognition is a field of computer vision which has shown great progress in the past decade. Starting from simple single person activities, research in activity recognition is moving towards more complex scenes involving multiple objects and natural environments. The main challenges in the task include being able to localize and recognize events in a video and deal with the large amount of variation in viewpoint, speed of movement and scale. This chapter gives the reader an overview of the work that has taken place in activity recognition, especially in the domain of complex activities involving multiple interacting objects. We begin with a description of the challenges in activity recognition and give a broad overview of the different approaches. We go into the details of some of the feature descrip- tors and classification strategies commonly recognized as being the state of the art in this field. We then move to more complex recognition systems, discussing the challenges in complex activity recognition and some of the work which has taken place in this respect. Finally, we provide some examples of recent work in complex activity recognition. The ability to recognize complex behaviors involving multiple Nandita M Nayak University of California, Riverside, 900 University Ave. Riverside, CA 92521, e-mail: nandita.nayak@email.ucr.edu Ricky J. Sethi University of California, Los Angeles, 4532 Boelter Hall, Los Angeles, CA 90095-1596, e-mail: rickys@sethi.org Bi Song University of California, Riverside, 900 University Ave. Riverside, CA 92521,e-mail: bsong@ee.ucr.edu Amit K. Roy-Chowdhury, University of California, Riverside, 900 University Ave. Riverside, CA 92521, e-mail: amitrc@ee.ucr.edu 1 This work has been partially supported by the DARPA VIRAT program and NSF award IIS- 0905671. 1