Increasing viewability, not backlighting, for displays

Increasing viewability, not backlighting, for displays

Most consumer devices suffer from poor sunlight viewability, but compensating with backlighting drains too much power

BY PAUL KARAZUBA
Senior Product Marketing Manager
QuickLogic
www.quicklogic.com

As display sizes on mobile phones and other consumer devices have grown, the importance of power consumption and display viewability has grown accordingly. With increasing numbers of features and functions crammed into smaller and smaller devices, batteries are taxed to levels never before seen, or perhaps even anticipated.

With displays often consuming as much as 50% of system power, system designers and device OEMs are constantly looking for ways to reduce the power consumption of the display without any impact to the user’s experience. Unfortunately, all previous accepted methods for display power consumption reduction have negative effects on the user’s viewing experience, especially when viewing video and mixed-media content.

Sunlight and batteries

One common theme among all portable devices is problems with display viewability in certain ambient-lighting environments, particularly bright sunlight. While simple monochrome E-ink-type reflective displays do not have this issue due to the high contrast ratio of black on white, popular display technologies like LCDs suffer from significant display viewability drop-off as reflected ambient-light levels increase. In some cases, display content will completely washout in bright sunlight, making viewing impossible and forcing the user to either find a lower-lighting environment, or wait until ambient lighting changes enough to allow content viewing.

The common solution for addressing this is to simply increase the display brightness, but this has a negative effect on user experience; namely, battery life suffers greatly.

As a rule of thumb, as the size of the display increases, so does power consumption. For example, a typical 4-in. LCD mobile phone display may consume as much as 650 mW at 100% backlight. However, a 10.1-in. LCD display like those used in tablet PCs can consume more than 2,000 mW at maximum backlight. While battery capacity certainly will increase when changing form factors (mobile phone to tablet), battery capacity will typically scale linearly with device size, meaning that single-charge battery life generally does not change from a 4-in. mobile phone to a 10-in. tablet.

Dynamic compensation’s role

All consumer devices employ dynamic range compensation (DRC) to render a source image or video stream suitable for display on an output device. Dynamic range is broadly defined as the difference in intensity between the darkest and brightest part of a scene. The human eye can capture a very wide dynamic range five orders of magnitude. However, typical displays can reproduce information over a range of only a few hundred counts or less.

The dynamic range capability of a display is governed by the display technology (LCD, PDP, or OLED), the power or brightness, the amount of screen reflection, and the ambient lighting conditions. Therefore, to retain as much information from a real-world scene after digital capture, transmission, and display, dynamic range must be compressed at each step along the chain. In some applications, dynamic range expansion may be required.

If the display is large enough that the video fills the viewer’s visual field, the eye can apply this processing itself. However, the most common use case in portable devices is that the display fills only the central portion of the field of view. The eye cannot perform this processing optimally and therefore digital processing must be substituted.

So in the case of viewing video on a mobile device, even if the display has a dynamic range capability equal to or exceeding that of the original video, DRC is still required to produce natural-looking video. Because the human eye can apply very strong DRC, a displayed video will only look natural and realistic if the same kind of processing is applied.

As both display power consumption and viewability have been known problems for many years, a number of DRC technologies have been developed to address them, in particular:

• Content adaptive backlight control (CABC)

• Ambient light compensation (ALC)

• Histogram equalization (HE)

• Visual Enhancement Engine (VEE) and Display Power Optimizer (DPO)

Adapting to content

CABC is a method of DRC whereby the display’s dynamic range capabilities are mapped to the content’s dynamic range. Most video and display content is intended to be viewed on displays with a dynamic range exceeding 1,000:1. However, most handheld consumer device displays have a dynamic range of ~600:1 (see Fig. 1 ). Note that this dynamic range is stated while in ideal viewing conditions, such as a dark room similar to a movie theater.

Fig. 1: Most video content has been created for displays with high dynamic ranges, and so must be altered dynamically to suit displays with lower ranges like those in a typical mobile device display.

Most video content will not actually have a dynamic range that exceeds 1,000:1. CABC will calculate the ratio of the video contents dynamic range to the intended displays dynamic range, and then scale the video content using that ratio to the mobile display (see Fig. 2a ). What results is video content that is scaled to match the original content versus display ratio.

Fig. 2: CABC looks to scale content based on the relative differences in range been display types.

CABC can then reduce the backlight level of the display to eliminate the upper end of the “wasted range” (see Fig. 2b ). However, the lower end remains wasted. The net result is that CABC has reduced display power consumption. Evidence shows that CABC can reduce system power consumption by as much as 10% to 15%.

But while CABC can save system power, it has a number of drawbacks:

1. Actual power savings are content-dependent and are thus hard to predict. While the display’s dynamic range does not change, the content’s dynamic range is always changing. CABC is constantly sampling and scaling, leading to actual power savings that are difficult, if not impossible, to predict. Content with a high dynamic range, such as movies or television shows, will lead to lesser power savings than low dynamic range content, such as children’s cartoons.

2. CABC does not account for ambient light. CABC looks at a display’s dynamic range under ideal viewing conditions, but the reality of mobile devices is that content is almost never viewed in ideal conditions. As ambient light increases, effective display dynamic range worsens. So as ambient-light levels increase, the viewability of the display worsens (see Fig. 2c ).

3. Changes to backlight levels can show up as flickering. CABC is constantly adjusting display brightness to display content. As the content’s dynamic range increases and decreases, the display will adjust brightness levels. If content is constantly changing dynamic range, display brightness will change just as much, which can result in a “flickering” effect to the viewer. At minimum, this effect is annoying; at maximum, it causes some viewers to stop using a device.

ALC and HE

Ambient-light compensation (ALC) is a very simple method of display power savings where the display backlight intensity is changed according to the ambient-light conditions. As the ambient-light level increases, display brightness increases, and as ambient light decreases, display brightness decreases. ALC algorithms are so simple that they tend to be included in the mobile device operating system programming available to OEMs. ALC does not adjust display content at all—only the display backlight is changed. As ambient light increases, details in the dark are lost first, and then more and more content is lost (see Fig. 3 )

Fig. 3: Ambient light compensation increases the backlight intensity, but as the ambient-light level increases, more of the detail disappears as contrast differences are lost because the content is not adjusted to compensate.

Histogram equalization (HE) is a simple method of adjusting the signal based on the range of the content, not the original display (see Fig. 4 ). HE increases the global contrast of the display content, especially where the display content consists of close contrast values. Through this adjustment, the intensities can be better distributed on the histogram (and thus the display). This allows for areas of lower local contrast to gain a higher contrast. Histogram equalization accomplishes this by effectively spreading out the most frequent intensity values.

Fig. 4: Histogram equalization brings content in line with the dynamic range of a mobile display, but does not discriminate between noise and signal.

While HE is a known method of image enhancement, it is widely considered to not be ideal for video and photographs.

• HE algorithms are indiscriminate. HE does not identify “good” versus “bad” content. It may increase the contrast of background noise, while decreasing the usable signal.

• HE can produce undesirable effects like visible image gradient when applied to images with low color depth.

VEE and DPO

Based on Apical Ltd’s iridix algorithm, QuickLogic’s Visual Enhancement Engine (VEE) and Display Power Optimizer (DPO) technologies implement a DRC method that applies different tonal and color transformations to every pixel in an image. Based on a model of human perception, the DRC algorithms results in a displayed image that retains detail, color, and vitality even under difficult viewing conditions.

VEE and DPO technologies specifically address the problem of the low contrast ratio of mobile displays to bring a more TV-like viewing experience to mobile devices while conserving power. In effect, VEE automatically generates and applies a different iridix tone curve transform to every pixel in the input video, based on global user parameters that control its general behavior.

The principal limitation to the strength of VEE processing is the source video’s signal-to-noise ratio. The basic algorithm does not distinguish between video detail and noise; noise in very dark or bright areas may be rendered visible after processing. Practical implementations of iridix tone matching include a gain control feature that limits the strength of processing in different intensity ranges, so that noise is always kept outside the visible range.

More recent additions to the core VEE algorithms have been the incorporation of modules for nonlinear space-variant color correction, noise reduction, and preservation of fine detail. To further improve display quality, VEE technology has been supplemented by extra image and video enhancement blocks such as dithering, hue rotation, color correction, and nonlinear sharpness filtering.

Table 1: Comparison of original and VEE-enabled efficiency

VEE and DPO have been measured in real-world applications. Table 1 shows the viewing experience and power consumption of two consumer devices using VEE and DPO. VEE effectiveness can be seen when the original backlight level is compared to the VEE-enabled backlight level. For the 5-in. display smartphone, in the outdoor use case, the VEE-off backlight level was set at 93%. When VEE is enabled, the OEM deemed that the display content quality was equal at a backlight level of 26%. The drop from 93% to 26% in backlight intensity allows for a DPO-based single-charge battery life improvement of 23%. ■

■