234 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 2, APRIL 2014 Assistive Clothing Pattern Recognition for Visually Impaired People Xiaodong Yang, Student Member, IEEE, Shuai Yuan, and YingLi Tian, Senior Member, IEEE Abstract—Choosing clothes with complex patterns and colors is a challenging task for visually impaired people. Automatic cloth- ing pattern recognition is also a challenging research problem due to rotation, scaling, illumination, and especially large intraclass pattern variations. We have developed a camera-based prototype system that recognizes clothing patterns in four categories (plaid, striped, patternless, and irregular) and identifies 11 clothing col- ors. The system integrates a camera, a microphone, a computer, and a Bluetooth earpiece for audio description of clothing patterns and colors. A camera mounted upon a pair of sunglasses is used to capture clothing images. The clothing patterns and colors are described to blind users verbally. This system can be controlled by speech input through microphone. To recognize clothing pat- terns, we propose a novel Radon Signature descriptor and a schema to extract statistical properties from wavelet subbands to capture global features of clothing patterns. They are combined with local features to recognize complex clothing patterns. To evaluate the ef- fectiveness of the proposed approach, we used the CCNY Clothing Pattern dataset. Our approach achieves 92.55% recognition accu- racy which significantly outperforms the state-of-the-art texture analysis methods on clothing pattern recognition. The prototype was also used by ten visually impaired participants. Most thought such a system would support more independence in their daily life but they also made suggestions for improvements. Index Terms—Assistive system, clothing pattern recognition, global and local image features, texture analysis, visually impaired people. I. INTRODUCTION B ASED on statistics from the World Health Organization (WHO), there are more than 161 million visually impaired people around the world, and 37 million of them are blind [9]. Choosing clothes with suitable colors and patterns is a challeng- ing task for blind or visually impaired people. They manage this task with the help from family members, using plastic braille la- bels or different types of stitching pattern tags on the clothes, or by wearing clothes with a uniform color or without any patterns. Manuscript received April 4, 2013; revised July 9, 2013, October 20, 2013, and January 7, 2014; accepted January 17, 2014. Date of publication February 13, 2014; date of current version March 12, 2014. This work was supported in part by the National Science Foundation under Grant IIS-0957016, Grant EFRI-1137172, the National Institutes of Health under Grant 1R21EY020990, Army Research Office Grant W911NF-09-1-0565, the Federal Highway Ad- ministration Grant DTFH61-12-H-00002, and Microsoft Research. This paper was recommended by Associate Editor Y. Yuan. X. Yang and Y. L. Tian are with the City College, City University of New York, New York, NY 10031 USA (e-mail: xyang02@ccny.cuny.edu; ytian@ ccny.cuny.edu). S. Yuan is with Teledata Communications Inc., Islandia, NY 11749 USA (e-mail: syuan00@ccny.cuny.edu). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/THMS.2014.2302814 Automatically recognizing clothing patterns and colors may im- prove their life quality. Automatic camera-based clothing pat- tern recognition is a challenging task due to many clothing pat- tern and color designs as well as corresponding large intraclass variations [4]. Existing texture analysis methods mainly focus on textures with large changes in viewpoint, orientation, and scaling, but with less intraclass pattern and intensity variations (see Fig. 1). We have observed that traditional texture analysis methods [3], [5], [10], [11], [15], [19], [23], [26], [29], [32] can- not achieve the same level of accuracy in the context of clothing pattern recognition. Here, we introduce a camera-based system to help visually impaired people to recognize clothing patterns and colors. The system contains three major components (see Fig. 2): 1) sen- sors including a camera for capturing clothing images, a micro- phone for speech command input and speakers (or Bluetooth, earphone) for audio output; 2) data capture and analysis to per- form command control, clothing pattern recognition, and color identification by using a computer which can be a desktop in a user’s bedroom or a wearable computer (e.g., a mini-computer or a smartphone); and 3) audio outputs to provide recognition results of clothing patterns and colors, as well as system status. In an extension to [30], our system can handle clothes with complex patterns and recognize clothing patterns into four cat- egories (plaid, striped, patternless, and irregular) to meet the basic requirements based on our survey with ten blind partic- ipants. Our system is able to identify 11 colors: red, orange, yellow, green, cyan, blue, purple, pink, black, grey, and white. For clothes with multiple colors, the first several dominant colors are spoken to users. In order to handle the large intraclass varia- tions, we propose a novel descriptor, Radon Signature, to capture the global directionality of clothing patterns. The combination of global and local image features significantly outperforms the state-of-the-art texture analysis methods for clothing pattern recognition. We also show that our method achieves compara- ble results to the state-of-the-art approaches on the traditional texture classification problems. This paper is organized as follows. In Section II, we sum- marize the related work on assistive techniques for visually impaired people and the research work on texture analysis. The computations of global and local features for clothing pattern recognition are described in Section III. Section IV introduces the system and interface design. The details of clothing pat- tern recognition and color identification are demonstrated in Section V. Section VI presents our experimental results on a challenging clothing pattern dataset and a traditional texture dataset. Section VII describes the preliminary evaluations by blind users. Section VIII concludes the paper. 2168-2291 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.