J.A. Jacko (Ed.): Human-Computer Interaction, Part II, HCII 2009, LNCS 5611, pp. 208–215, 2009. © Springer-Verlag Berlin Heidelberg 2009 A Real-Time Hand Interaction System for Image Sensor Based Interface SeIn Lee 1 , Jonghoon Seo 1 , Soon-bum Lim 2 , Yoon-Chul Choy 1 , and TackDon Han 1 1 Dept. Of Computer Science, Yonsei University, 134, Sinchon-dong, Seodaemun-gu {sipkjy,jonghoon.seo,ycchoy,hantack}@yonsei.ac.kr 2 Dept. Of Multimedia Science, Sookmyung University, 52 Hyochangwon-gil, Yongsan-gu, Seoul, Korea sblim@sookmyung.ac.kr Abstract. Diverse sensors are available in ubiquitous computing of which re- sources is inherent in environment. Among them, image sensor acquires neces- sary data using camera without any extra devices, which is a different aspect from other sensors. It can provide additional services and/or applications by us- ing a location of code and ID in real time image. Focusing on this, Intuitive in- terface operating method in ubiquitous computing environment that has plenty of image codes is suggested. GUI using image sensor was designed, which works real-time interactive operation between user and the GUI without any additional button or device. This interface method recognizes user's hand im- ages in real-time by learning them at a starting point. The method sets interac- tion point, and operates the GUI through hand gestures defined previously. We expect this study can be adopted to augmented reality area and real time inter- face using user’s hand. Keywords: HCI, Hand interface, Hand interaction, Augmented Reality. 1 Introduction In 2008, Microsoft’s Bill Gates emphasized the importance of convenient interfaces using the phrase “second digital decade.”[1] However, there are many inconveniences with the interface. Input inconvenient and lacking in intuition, thus necessitating additional learning. In contrast, the second digital decade interfaces need to be more intuitive, user-friendly, and convenient to use. We suggest an interaction system that satisfies the needs for future ubiquitous computing environment. Various information and applications can be used in ubiquitous computing environment. The ubiquitous computing environment is assumed for the study, in which digital information used has a 2d image code form. We are considering image sensor based real time and real environment service or application under the environment. This is a kind of augmented reality (AR) approach. In this paper, as a method to interact with GUI in AR environment, we present a system that directly uses user's hand.