Search Results for « Gesture-based »

Increasing numbers of everyday objects in libraries, stores and warehouses are instrumented with passive RFID tags, resulting in a ripe opportunity for gesture-based interactions with people. By a simple act of picking up and gesturing with an RFID-tagged object, users can send their opinions and sentiments about that object to the cloud. Prior work in RFID-based gesture tracking relies on multiple bulky and expensive antennas and readers to function, which incurs unacceptable infrastructure costs for large-scale ubiquitous deployment (over an entire warehouse or mall, for example) thus hindering practical adoption. In this paper, we propose Pantomime, the first RFID-based gesture recognition system that uses just a single antenna per geographical area of coverage. Our key insight is to replace the conventional multiple antenna single tag tracking framework with an equivalent multiple tag single antenna system. Through a novel tag coordination protocol and a lightweight tracking algorithm, Pantomime enables accurate gesture tracking that works for objects tagged with just two RFID tags. We implement a real-time prototype of Pantomime with commercial off-the-shelf (COTS) RFID readers and antennas. Extensive evaluations and real-world case studies in a classroom and a retail store demonstrate that Pantomime achieves comparable gesture tracking accuracy (87%) to state-of-the-art multi-antenna schemes (88%) at a minimal deployment cost.

This paper presents two probabilistic developments for the use with electromyograms (EMGs). First described is a neuroelectric interface for virtual device control based on gesture recognition. The second development is a Bayesian method for decomposing EMGs into individual motor unit action potentials (MUAPs). This Bayesian decomposition method allows for distinguishing individual muscle groups with the goal of enhancing gesture recognition. All examples presented rely upon sampling EMG data from a subject's forearm. The gesture-based recognition uses pattern recognition software that has been trained to identify gestures from among a given set of gestures. The pattern recognition software consists of hidden Markov models, which are used to recognize the gestures as they are being performed in real time from moving averages of EMGs. Two experiments were conducted to examine the feasibility of this interface technology. The first replicated a virtual joystick interface, and the second replicated a keyboard. Moving averages of EMGs do not provide an easy distinction between fine muscle groups. To better distinguish between different fine motor skill muscle groups, we present a Bayesian algorithm to separate surface EMGs into representative MUAPs. The algorithm is based on differential variable component analysis, which was originally developed for electroencephalograms. The algorithm uses a simple forward model representing a mixture of MUAPs as seen across multiple channels. The parameters of this model are iteratively optimized for each component. Results are presented on both synthetic and experimental EMG data. The synthetic case has additive white noise and is compared with known components. The experimental EMG data were obtained using a custom linear electrode array designed for this study.

This chapter surveys the main methods and techniques for gesture-based computing. We consider both mid-air and touch-based gestures. The survey includes techniques for gesture recognition (both hardware and software) and the most widespread applications in the field. Here, we summarize the state-of-the-art of the field, outline a brief history of its development, and identify future research challenges.

The use of doctor-computer interaction devices in the operation room (OR) requires new modalities that support medical imaging manipulation while allowing doctors' hands to remain sterile, supporting their focus of attention, and providing fast response times. This paper presents "Gestix," a vision-based hand gesture capture and recognition system that interprets in real-time the user's gestures for navigation and manipulation of images in an electronic medical record (EMR) database. Navigation and other gestures are translated to commands based on their temporal trajectories, through video capture. "Gestix" was tested during a brain biopsy procedure. In the in vivo experiment, this interface prevented the surgeon's focus shift and change of location while achieving a rapid intuitive reaction and easy interaction. Data from two usability tests provide insights and implications regarding human-computer interaction based on nonverbal conversational modalities.

The goal of this research is to explore new interaction metaphors for augmented reality on mobile phones, i.e. applications where users look at the live image of the device’s video camera and 3D virtual objects enrich the scene that they see. Common interaction concepts for such applications are often limited to pure 2D pointing and clicking on the device’s touch screen. Such an interaction with virtual objects is not only restrictive but also difficult, for example, due to the small form factor. In this article, we investigate the potential of finger tracking for gesture-based interaction. We present two experiments evaluating canonical operations such as translation, rotation, and scaling of virtual objects with respect to performance (time and accuracy) and engagement (subjective user feedback). Our results indicate a high entertainment value, but low accuracy if objects are manipulated in midair, suggesting great possibilities for leisure applications but limited usage for serious tasks.

We introduce the Gesture Watch, a mobile wireless device worn on a user's wrist that allows hand gesture control of other devices. The Gesture Watch utilizes an array of infrared proximity sensors to sense hand gestures made over the device and interprets the gestures using hidden Markov models. The Gesture Watch maps intuitive gross hand gestures to control signals such as the play and pause commands commonly found on mobile media players. We present our evaluation of the Gesture Watch designed to determine its the accuracy and usability. In our study, 10 participants used the Gesture Watch in both mobile and stationary conditions as well as indoors and outdoors. Overall, we attained a recognition accuracy of 95.5% and found that the Gesture Watch worked well in both indoor and outdoor environments and while mobile.

As gesture interfaces become more main-stream, it is increasingly important to investigate the behavioral characteristics of these interactions – particularly in three-dimensional (3D) space. In this study, Fitts’ method was extended to such input technologies, and the applicability of Fitts’ law to gesture-based interactions was examined. The experiment included three gesture-based input devices that utilize different techniques to capture user movement, and compared them to conventional input technologies like touchscreen and mouse. Participants completed a target-acquisition test and were instructed to move a cursor from a home location to a spherical target as quickly and accurately as possible. Three distances and three target sizes were tested six times in a randomized order for all input devices. A total of 81 participants completed all tasks. Movement time, error rate, and throughput were calculated for each input technology. Results showed that the mean movement time was highly correlated with the target's index of difficulty for all devices, providing evidence that Fitts’ law can be extended and applied to gesture-based devices. Throughputs were found to be significantly lower for the gesture-based devices compared to mouse and touchscreen, and as the index of difficulty increased, the movement time increased significantly more for these gesture technologies. Error counts were statistically higher for all gesture-based input technologies compared to mouse. In addition, error counts for all inputs were highly correlated with target width, but little impact was shown by movement distance. Overall, the findings suggest that gesture-based devices can be characterized by Fitts’ law in a similar fashion to conventional 1D or 2D devices.