The Visual Computer
https://doi.org/10.1007/s00371-019-01758-8
ORIGINAL ARTICLE
Gestural flick input-based non-touch interface for character input
Md. Abdur Rahim
1
· Jungpil Shin
1
· Md. Rashedul Islam
1
© Springer-Verlag GmbH Germany, part of Springer Nature 2019
Abstract
A non-touch character input is a modern system for communication between humans and computers that can help the user to
interact with a computer, a machine, or a robot in unavoidable circumstances or industrial life. There have been many studies
in the field of touch and non-touch character input systems (i.e., hand gesture languages), such as aerial handwriting, sign
languages, and the finger alphabet. However, many previously developed systems require substantial effort in terms of learning
and overhead processing for character recognition. To address this issue, this paper proposes a gesture flick input system that
offers a quick and easy input method using a hygienic and safe non-touch character input system. In the proposed model, the
position and state of the hands (i.e., open or closed) are recognized to enable flick input and to relocate and resize the on-screen
virtual keyboard for the user. In addition, this system recognizes hand gestures that perform certain motion functions, such
as delete, add a space, insert a new line, and select language, an approach which reduces the need for recognition of a large
number of overhead gestures for the characters. To reduce the image-processing overhead and eliminate the surrounding noise
and light effects, body index skeleton information from the Kinect sensor is used. The proposed system is evaluated based
on the following factors: (a) character selection, recognition and speed of character input (in Japanese hiragana, English, and
numerals); and (b) accuracy of gestures for the motion functions. The system is then compared to state-of-the-art algorithms.
A questionnaire survey was also conducted to measure the user acceptance and usability of this system. The experimental
results show that the average recognition rates for characters and motion functions were 98.61% and 97.5%, respectively,
thus demonstrating the superiority of the proposed model compared to the state-of-the-art algorithms.
Keywords Human–computer interaction · Hand gestures · Non-touch input · Gesture recognition · Kinect sensor
1 Introduction
Humans communicate with computers in many ways, and the
interface between humans and computers is an important fac-
tor in the convenience of this interaction. Most of the research
in this field therefore aims to improve the usability of the com-
puter interface and to enhance human–computer interactions
[1]. In the last few decades, the keyboard and mouse have
played important roles in human–computer interactions. At
present, the user is expected to use both touch and non-touch
interfaces to interact with the machine, and various differ-
B Jungpil Shin
jpshin@u-aizu.ac.jp
Md. Abdur Rahim
rahim_bds@yahoo.com
Md. Rashedul Islam
rashed.cse@gmail.com
1
School of Computer Science and Engineering, The University
of Aizu, Fukushima 965-8580, Japan
ent devices are used in daily life, such as computers, tablet
PCs, and smartphones. A keyboard is typically used to input
characters on computers, and software keyboards to input
characters on smartphones or tablet PCs. The user needs to
touch the keyboard to input characters, even on smartphones
or tablet PCs. There is also a large demand in industry to
provide a hygienic and safe environment, and rugged and
reliable computing and automation devices may be needed
to increase production. These tough environments and tight
regulations require special equipment to keep the environ-
ment sterile and safe. In order to meet the highest hygiene
and safety requirements, to ensure the traceability of all pro-
cessing steps and to ensure the implementation of existing
standards, many industrial applications have used advanced
human–machine interface (HMI) technology [2]. There are
many industrial computer production companies (e.g., JLT
mobile computers, Noax Technology, Teguar Computer, and
others) that offer fully sealed, waterproof, industrial touch-
screen automation computers for production applications to
123