Touching air: Wake your device with a simple gesture

Touching air: Wake your device with a simple gesture

BY ILYA VEYGMAN, Strategic Applications Engineer
Maxim Integrated Products
www.maxim-ic.com

This article discusses how to wake up a touchscreen-based device such as a tablet…without a touch. Instead you use a very basic form of gesture recognition and a novel proximity sensor. Topics discussed include physical layout, speed limitations, detection thresholds, high-level system integration, and the “human” factor. Example code illustrates software implementations.

Cooking up some wild ideas

If you ever used a touchscreen device while cooking, you probably noticed that following a recipe from your device is not as easy as it seems. Tech-savvy cooks, such as yours truly, like to use tablet computers or smartphones to reference recipes while making dinner. “Fine,” you say, “but what is so challenging about that?”The handheld device normally goes to sleep after a minute or two, since keeping the screen on can burn up considerable power. Then when you want to reference something and the device is asleep, you are faced with two choices: either force the screen to stay on permanently, or risk smudging the screen with food-stained hands. There’s always the option of washing one’s hands every time that something has to be checked, but constant washing and drying is tedious and wastes water.

This is where I ask myself, “Can I avoid keeping the screen on permanently and also avoid the risk of fouling the device?” There is, actually, a way to do both. You can gesture for the screen to come back on without actually touching it. Seems complicated, right? Fortunately, this is simpler to do than it sounds.

Getting close to proximity sensors

Many touchscreen devices, especially smartphones, have infrared (IR) proximity sensors built in. These sensors are generally used to turn the screen on and off during a call and, thus, prevent undesired input from being applied to the phone. This sensor, and some clever software design, is all you need to make a touch-free wakeup operate with the wave of a hand.

The basic idea is when the device is asleep, i.e., when its touch screen is turned off and the applications processor is in a low-power mode, have the proximity sensor “look” for a large enough change from the background readings and react appropriately. This is almost exactly how the proximity sensor already operates to turn off the screen during a phone call. In our application, the data is simply interpreted somewhat differently.

You start by noting the amount of “normal” background counts read by the sensor. This may be zero counts, although it is useful to account for systemic offsets (e.g,. backscatter or crosstalk), when applicable. Then set it up to either trigger an interrupt or send a signal to the applications processor when the signal climbs past the threshold that was set. This will then cause the system to come back online and turn on the screen. Overall, this is quite straightforward and can be implemented with an ambient light and IR proximity sensor.

This demonstration uses the MAX44000, which takes a proximity reading every 1.56ms or as slow as every 100ms (when interleaved with the ambient light sensor). Assuming a maximum detection range of 10cm and an LED with a radiation angle of ±15°, then the coverage area is about 22cm2 or approximately 5.35cm across. A target moving across this screen area must be sampled at least once to be seen. Hence, the fastest gesture that can be reliably sensed, or “picked up,” in the slowest and also lowest power sampling speed is about 0.53mps. We are also assuming here that the sensor needs to sense only one sample above a threshold for it then to recognize a simple target passing through the coverage area.

It’s all in the wrist…

As a theoretical approach, this is very simple. When the device enters sleep mode, set its proximity sensor to scan the environment and send an interrupt when it detects a target, indicated by the signal moving above a preset threshold. This can be done by simply polling the sensor repeatedly over the I2C interface. Unfortunately, this can also consume more power than most operators would like.

This is the point at which the specific capabilities of your proximity sensor become most important. The MAX44000 sensor is designed to let the applications processor take a more hands-off (and lower power) approach.

By enabling the MAX44000’s internal proximity interrupts (bit 1 in register 0x01), you can write the wakeup threshold to an internal register (0x0B and 0x0C). When the proximity reading passes this threshold, an interrupt flag is raised. This drives the MAX44000’s active-low INT pin low, which indicates an interrupt on the external line. When the applications processor sees that this line has been driven low, it can be either awakened to bring the device out of its low-power state and bring the screen back online, or it can do whatever else needs to be done.

…But let’s not hand-wave through this

As is often true, the practical application is not as easy as the theoretical. Unfortunately, hands-free wakeup is not quite as simple as seeking one sample above a threshold. Instead, there are several factors to consider when attempting to implement this design.

Signal level and layout

Perhaps the most critical consideration is the signal level chosen to trigger a wakeup condition. An important trade-off exists between the responsiveness of the system and its immunity to false detection. Setting the threshold low makes it easier to detect an input (e.g., your waving hand), but increases the risk of false detection from transient noise or incident motion. Conversely, too high a threshold will almost surely reduce the likelihood of a false detection to zero, but it can also cause the system to only detect very close targets or even be unresponsive to most any input (e.g., your frantic waving hand).

The best way to address this is, first, to mitigate the amount of noise in the system. This can be done by optical solutions or careful electrical routing and component placement. By lowering the noise floor, you reduce the chances of a false detect. Next, choose an “average” detection distance (e.g., 4cm to 5cm) and measure the signal with a benchmark target. An 18% gray card is ideal. Note that it might seem obvious, but if the application uses a dark glass in front of the sensor, then the measurements should be done with that glass in place. The signal level measured can be used as a best guess for setting the threshold. A good rule is to set the level at 8% to 15% of full scale, although your level can vary.

The MAX44000 sensor’s proximity threshold registers let you set a wakeup threshold based on the above experiment. Figure 1shows a signal versus distance plot for an 18% grey card with a 100mA drive current and no glass in front of the sensor. The blue line is a possible choice for wakeup threshold.

Figure 1 .Distance vs. signal strength using the MAX44000 proximity sensor with an 18% grey card, 100mA current, and no glass.

Noise and lowpass Filtering

If noise is still a concern, you can implement a lowpass filter to clean up the signal. There are two ways to do this with the MAX44000 because it also has a few bits to enable threshold persistence before an interrupt flag is raised. This setting, which requires the proximity reading to stay outside the threshold for a set number of samples, can be used to reduce the effects of noise.

A slightly more complex method involves storing the readings from the sensor in a data queue, then applying a customized FIR filter to them in software. Unfortunately, this approach has a flaw. If the sampling rate of the proximity sensor cannot be increased, then the rate at which you can swipe your hand through the sensor’s field of view and still be detected is reduced. The 100ms effective sampling rate is particularly vulnerable to this. When using the persistence threshold, this reduced detection can be as much as 16x (although the 4x persist should be enough).

Wave speed

This brings us to the next consideration, wave speed. The maximum speed is determined by first, the sensor’s field of view; second, the distance of the wave from the sensor; third, the sampling rate; and fourth, the threshold level. Determining the first two criteria is obvious: the angle that the sensor can see, coupled with the distance of the target from the sensor, can be used with some basic trigonometry to calculate the distance which the target can travel while the sensor tries to pick it up. For example, if the sensor’s viewing angle is 30 degrees in total and the valid range is up to 10cm, then the target can travel 5.35cm over the sensor and still be seen. This is a coverage area of about 78cm2. This linear distance, coupled with the sampling rate, implies a speed limit.

Specifically, if the sampling rate is T, then the target must cross the visible area in no less than T. For example, if T is 100ms (the MAX44000’s slowest rate), then following the previous example, the maximum speed theoretically allowed is 1mps. (This is actually quite fast.) You might want to capture multiple samples to confirm your assessment, so this speed limit will likely be reduced.

The detection threshold also plays a part in the maximum speed allowed. As a rule, the lower the threshold, the faster the gesture that can be picked up. As mentioned above, this low threshold should be carefully selected to avoid false detections.

The “Human” Factor

This application is subject to the inconsistencies of the human hand and the person waving it. Consequently, the design ought to involve use cases to determine how fast typical users move their hands in front of the screen, their distance from the screen, and whether they are wearing gloves. This interaction with a device can also vary greatly among applications like smartphones, tablets, or something on an automobile dashboard. Ultimately, the application must account for these user-interface and experience variables during the design process.

There is one final thing to consider, when is a wave not a wave? To be more specific, how can a device know if a received signal comes from a waving hand or another simple movement like being placed inside a case; a pocket or backpack, or face-down on some surface? We need to face the truth here—there is no easy answer to this question, unless context is given to the device. How that is done is a different matter entirely.

In the end, you can choose to implement this wakeup solution only when the touchscreen device is running certain applications or you can require the user to enable it Additionally, many of these devices have accelerometers that can tell the device when it is lying face-down. The feature can also be disabled if the device was put to sleep manually by the user (e.g., when the device is turned off).

Try It Yourself

For your convenience, three pieces of sample code are attached to this article. The first implements a conceptually simple version of this wakeup solution using manual checks of the MAX44000’s proximity reading. The second code expands on the first and implements the filtering concept discussed earlier. The last code shows a quick way of doing this wakeup using the MAX44000’s interrupt feature.

Fig. 1 Example Code 1

Fig. 2 Example Code 2

Fig. 3 Example Code 3