UIST 09- Tounge Gestures

T. Scott Saponas, Daniel Kelly, and Babak A. Parviz (University of Washington, Seattle) and Desney S. Tan (Microsoft Research)
Summary

This paper is aimed at exploring new ways of interaction for those with severe physical handicaps. In most cases of gross motor loss the ability to control eyes, jaw and tongue often are unaffected. Much work has been done exploring eye tracking and speech recognition; this paper attempts to explore the untouched area of tongue input. Prior work on tongue sensing have used things like a mini joystick in your mouth that can be controlled with the tongue or pressure sensitive buttons on a dental retainer. This paper asserts that these devices treat the tongue only like a finger when in fact the tongue is a complex muscle able often used to perform feats of dexterity like swallowing or generating speech. The device they created is an optical tongue sensing retainer (pictured above). Their optical approach allows for tongue gesture recognition. In their experiment they used four gestures, left swipe, right swipe, tap on the palette, hold on the palette. The qualitative results of the experiment exposed some interesting tongue-unique problems. Gesturing with the tongue causes it to deform and change shape. The set of tongue gestures they implemented could be performed in multiple ways ie. a swipe against your front teeth or a swipe in the back of your mouth. Also the varying difference in mouth and tongue shape across participants required a large amount of custom configuring and calibration.

Thoughts

Although I was not aware about tongue sensing technologies, after hearing about the approaches of prior work I was intrigued by this paper's view of the tongue as a unique modality instead of as a finger. I was disappointed however by the non innovative gestures. Swipes, taps and holds seem to me to be finger-centric designs. After they defined the tongue as a unique non-finger-like control I was hoping for some more innovative tongue gestures. I'm not sure, but I would maybe use the tongue gestures that you naturally use when speaking. That way you could map gestures to different sounds like 'eeh' 'ooh' 'sst' 'ess' 'mmm' etc. I think this would be far more intuitive and useful.