Spin-off aims to increase ‘gesture vocabulary’ of touchscreens

At November's MEMS Executive Congress in Napa, CA, a new capability that promises to increase the number of gestures that people can use to interact with touchscreens was demonstrated. The technology was developed by Julia Schwarz and Chris Harrison as a computer-human-interaction research project when they were graduate students at Carnegie Mellon University. Schwarz and Harrison have recently created a spin-off, Qeexo, to bring the technology to the commercial world.

While today's touchscreens can recognize simple taps and pinches performed by the finger, the screens are actually insensitive to the nature of the touch; you can touch with the thumb or index finger and, to the device, it's all the same. However, Schwartz and Harrison recognized that the manner in which a screen is touched can have a different vibrational, or acoustic, signature that can be used to differentiate between the ways in which the screen is touched. They set about determining the unique vibrational characteristics that occur when a screen is touched by say a knuckle or a fingernail, and created algorithms capable of distinguishing between them.

In addition to traditional finger taps, Qeexo technology can differentiate between touches with a fingernail, knuckle, stylus, or eraser. This allows apps to present different options depending on the type of touch sensed (lower right). 

So far, the duo have demonstrated the ability to differentiate between touches made with a fingertip, knuckle, fingernail, stylus, and pencil eraser, thereby increasing the gesture vocabulary of the touchscreen fourfold. In their demo at the MEMS Congress, they were able to show how the different types of touch could be used to perform a complex series of actions in way that is much simpler than when only basic touch is used. They modified a standard Samsung Galaxy smartphone by adding their software and fine tuning the phone’s existing sensors for the capability (See video).