Vol. 2 (2012) No. 1 ISSN: 2088-5334 A Color Based Touchless Finger Mouse Kah-Meng Kwong, Chin-Poo Lee, Kian-Ming Lim, Siew-Chin Chong, Siti Fatimah Abdul Razak Multimedia University, Jalan Ayer Keroh Lama, 75450 Bukit Beruang, Melaka, Malaysia. E-mail: kwong.kah.meng07;cplee;kmlim;chong.siew.chin;fatimah.razak@mmu.edu.my Abstract— People work with computers almost anytime, everywhere in the current trend. However, continuously controlling a computer with mouse for a long time might cause much strains to people’s wrist. This work proposes a touchless finger mouse using webcam. A marker with different colours representing different actions is used. The webcam will capture the information on the marker and trigger the associated actions. This prototype is proven to be able to perform most of the actions a normal mouser can perform. Keywords— Image processing; computer vision; finger mouse. I. INTRODUCTION Nowadays, computers have become an inherent part of human livings. People always deal with computers in their work, in their life, and even in their entertainment activities. In work, they use applications like word processing, spreadsheets, or some specialized software like Computer Aided Design (CAD), SPSS predictive analytics software, and so forth to solve their daily tasks. In life, people are shifting the trend to online transactions. Goods will be sent to their door by just a few clicks. Bills are paid by just a few clicks. Hang-outs with friends could be done via internet as well. Computers are just anytime, everywhere, everyone. A mouse is typically used with a computer to control the motion of a cursor in a graphical user interface (GUI). There are some main operations that a mouse can perform. Firstly, pointing-and-clicking can select files, programs or actions from a list of menus, or through icons. It can also trigger the floating menu which will only appear by right clicking. More particularly in clicking, a user can right click, single click (left click), double click, triple click, and etc. Secondly, drag-and-drop can move the object by holding the mouse button down while moving the cursor to different location. However, it is recently found that a mouse can cause repetitive strain injury (RSI) [1], [2] when users repetitively control a mouse to perform their tasks on the computer. In view of this, our work proposes a touchless finger mouse where the users are able to perform mouse operations simply by showing their finger in front of a webcam. The non- contact element makes this a good solution for outdoor installations because no contact with peripherals is needed. Section 2.0 explains some issues related to mouse and how the prototype in this work overcomes the issues. Section 3.0 describes the technical aspects of the prototype and ultimately some conclusions are drawn in Section 4.0. II. RELATED ISSUES There are some ergonomically designed mice to help increasing the comfort when using a mouse, such as a roller mouse, vertical mouse, joystick, or switching from using a mouse to a stylus pen with graphic tablet [3]-[5], but in chronic RSI they may result only in shifting the problem to another area. Some of these devices are expensive, which might not be widely affordable to general users. The proposed prototype is designed to address the above problems while preserving the main characteristics of a mouse. Firstly, it is simple to operate. Secondly, it is low cost device. Thirdly, it can select graphic object quickly. Lastly, it is used for quick positioning of cursor on an item. Besides that, the proposed work also has some strength over the existing devices to substitute mouse, as follows: a. For those existing free hand mouse control program, users may need to use some high cost unique devices [6]-[8] or high resolution cameras. As for the prototype, it just needs to use a low cost color marker which could be D.I.Y. and a webcam. b. Most of the existing free hand mouse control programs need some complex steps to perform a simple click operation. The prototype is able to perform click operations, drag operations, and some other simple actions. 66