ADL Recognition Based on the Combination of RFID and Accelerometer Sensing Maja Stikic * , Tˆ am Hu` ynh † , Kristof Van Laerhoven † and Bernt Schiele † * Fraunhofer IGD, Darmstadt, Germany † Computer Science Department, TU Darmstadt, Germany {stikic, huynh, kristof, schiele}@mis.tu-darmstadt.de Abstract—The manual assessment of Activities of Daily Living (ADLs) is a fundamental problem in elderly care. The use of miniature sensors placed in the environment or worn by a person has great potential in effective and unobtrusive long term monitoring and recognition of ADLs. This paper presents an effective and unobtrusive activity recognition system based on the combination of the data from two different types of sensors: RFID tag readers and accelerometers. We evaluate our algorithms on non-scripted datasets of 10 housekeeping activities performed by 12 subjects. The experimental results show that recognition accuracy can be significantly improved by fusing the two different types of sensors. We analyze different acceleration features and algorithms, and based on tag detections we suggest the best tags’ placements and the key objects to be tagged for each activity. I. I NTRODUCTION In order to fulfill the special needs of an increasing el- derly population, elderly care is becoming a rapidly growing problem, especially in western societies. The most common way of detecting the first changes in behaviour of an elderly person is monitoring of everyday activities that are usually performed on a daily basis. For that purpose, two specific sets of activities that describe the functional status of a person (Activities of Daily Living (ADL) - bathing, dressing, toileting, transferring, continence, feeding), as well as interaction with the physical and social environment (Instrumental Activities of Daily Living (IADL) - using telephone, shopping, food preparation, housekeeping, doing laundry, transportation, tak- ing medications, handling finances) have been defined [1]. The assessment of ADLs/IADLs is mostly done manually through interviews and questionnaires. As this is a time consuming and error prone process [2], it could highly benefit from automatic assessment technology. Various approaches to the ADL recognition problem can be found in literature. The two most common approaches are based on complementary assumptions. The first approach is based on the assumption that the objects people use during the execution of an activity robustly categorize the activity. In this approach, sensors are typically placed in the envi- ronment to detect user’s interactions with objects [3]. Radio Frequency Identification (RFID) tags and readers are usually used in these activity recognition systems, because of their durability, small size, and low costs. The second approach adopts the assumption that the activity is defined by motion of the body during its execution. Research in the wearable computing community has shown that characteristic movement patterns for activities such as running, walking or lying can effectively be inferred from body-worn accelerometers (e.g. [4], [5]). More specialized activities that have been recognized with accelerometers include household activities [4], physical exercises [6], wood workshop activities [7], and American Sign Language [8]. The goal of the research presented in this paper is to improve the recognition results by integrating these two ap- proaches, while also aiming to compensate for the shortcom- ings of both. In order to be able to accurately recognize different activities, the RFID approach requires a large number of objects to be tagged. However, we argue that it is not feasible to tag all objects, because of several reasons. First, the deployment of the large number of tags is still time consuming and error prone. Second, it is not practical to tag some objects because of their material (e.g. metal) or specific usage (e.g. objects used in microwave). We propose to use only the key objects for a specific set of activities by augmenting the object usage with a complementary sensing technique (i.e. accelerometers). On the other hand, the accelerometers approach requires multiple sensors to be placed on strategic body locations, such as wrist, hip, and thigh [4] for accurate recognition. We propose to use only a single 3D accelerometer at the dominant wrist of the user. Since the target user group of elderly people might not be familiar with modern information technologies, limiting the hardware to a single wrist-mounted device containing both the RFID tag reader and the accelerometer could increase user acceptance of our ADL monitoring system. There have been attempts to combine accelerometers with other sensor modalities, such as microphones (e.g. [7], [9]), wearable cameras [8] and recently, RFID tag readers [10]. We extend this promising approach [10] in the following directions: 1) Since we want to tag only a few important objects per activity, we perform the experiments with different numbers of tagged objects. Results of the experiments show that satisfactory recognition results can be achieved with fewer tagged objects than are usually used. 2) We evaluate our algorithms on a challenging multi-person dataset. The dataset is released and publicly available [11]. 3) We analyze different features and window lengths, as well as different ways of combining the activity recognition results from the two sensor modalities. Unlike in [10], our primary source of information is the RFID data, because it provides accurate