ORIGINAL RESEARCH Behavior modeling in industrial assembly lines using a wrist-worn inertial measurement unit Heli Koskima ¨ki • Ville Huikari • Pekka Siirtola • Juha Ro ¨ning Received: 30 March 2011 / Accepted: 19 July 2011 / Published online: 3 August 2011 Ó Springer-Verlag 2011 Abstract In this study an approach to an imperceptible and reliable worker monitoring system for industrial assembly lines is presented. A single wrist-worn inertial measurement unit is attached to the active wrist of the worker and by using acceleration and angular speed information, the behavior of the worker is recognized. The recognition is done in two steps. First the data are divided into 2-s intervals in which the performed activity is rec- ognized using a knn classifier so that the system is usable online. In the second step, a state machine is used to rec- ognize the completed tasks by searching for continuous, unvarying activity chains. The approach was developed as user-independent, although it can be easily adapted to a user-dependent case. By using the approach, behavior was recognized correctly and, on average, the correct beginning and ending moments of the behavior were missed by only 1 s. Thus a reliable monitoring system can be developed for industrial assembly lines. This work was supported by the EU 6th Framework Program Project XPRESS. Keywords Activity recognition Á Accelerometer Á State machine Á Manufacturing Á Proactive instruction Á Control 1 Introduction As the practical constraints related to wearable sensors have been solved, it is now possible to utilize the sensors more widely in various real-world applications. For example, they can be used to monitor workers on different industrial assembly lines. In this study, the emphasis is on finding continuous, unvarying activity chains to discover the total duration of performance of certain tasks. This information can be used to assure that the task is done correctly or the information can be used when developing proactive instruction systems. For example, if a worker has been using a spanner for more than 3 s, it can be assumed that the bolt is attached, after which a check list can be filled automatically or instructions for the next task can be given. The idea of the final monitoring system is to com- bine these purposes of use. The system monitors the worker, fills the check list automatically, informs the worker of incorrect tasks, and shows instructions proac- tively. In addition, the worker can control the system, for example by marking tasks as done manually. In this study, the term ‘behavior’ is used to describe an unvarying activity chain and certain behavior is considered necessary to complete a work task. For example, a bolt is being attached if ‘spanner use behavior’ is recognized. The data for behavior recognition were gathered using a single wrist-worn inertial measurement unit attached to a worker’s active wrist. Four different work tasks, hammer- ing, using a screwdriver, using a spanner, and attaching screws with a power drill, were performed as a sequence. Thus the data consisted of not only performance of the tasks, but also data where the worker moved around the post, changed the tool being used or just stayed still. The actual behavior recognition was divided into two different steps: activity recognition in certain time intervals H. Koskima ¨ki (&) Á V. Huikari Á P. Siirtola Á J. Ro ¨ning Computer Science and Engineering Laboratory, University of Oulu, P.O. Box 4500, Oulu 90014, Finland e-mail: hejunno@ee.oulu.fi V. Huikari e-mail: villehui@ee.oulu.fi P. Siirtola e-mail: pesiirto@ee.oulu.fi J. Ro ¨ning e-mail: jjr@ee.oulu.fi 123 J Ambient Intell Human Comput (2013) 4:187–194 DOI 10.1007/s12652-011-0061-3