Hybrid image sensor improves DMS

STMicroelectronics has introduced a new hybrid sensor that extends driver-monitoring systems (DMS) beyond assessing driver alertness to full interior monitoring of the driver and all passengers. The new hybrid rolling and global-shutter image sensor will enable new applications such as passenger safety-belt checks, vital-sign monitoring, child-left detection, gesture recognition, and high-quality video/picture recording.

(Source: STMicroelectronics)

The new sensor, the VD/VB1940, combines the sensitivity and high resolution of infrared sensing with high-dynamic-range (HDR) color imaging in a single component. The hybrid sensor can capture frames alternatively in rolling-shutter and global-shutter modes.

The 5.1-megapixel image sensor in an 1/2.5-inch optical format captures HDR color images for occupant monitoring systems (OMSs) and near-infrared (NIR) images typically captured by standard DMS sensors. NIR imaging can analyze driver head and eye movements in all lighting conditions.

The sensor captures NIR images in global-shutter mode, allowing synchronization with the infrared LED emitter to capture fast-moving scenes without motion blur. In rolling-shutter mode, which reads pixel data row by row, the VD/VB1940 is said to deliver optimized color-imaging performance. It delivers up to 60 frames per second at full resolution.

The automotive image sensor is built on ST’s second-generation 3D-stacked backside illuminated (BSI) wafer technology. This maximizes the optical area and on-chip processing in relation to die size and allows the sensor to perform sophisticated algorithms locally for optimal performance in color and NIR imaging, said ST. This results in power savings and offloads demand for an external co-processor.

Block diagram of the VD1940/VB1940 hybrid image sensor. Click for a larger image. (Source: STMicroelectronics)

Algorithms performed on-chip include Bayer conversion and HDR merging. With the on-chip HDR-merge feature in rolling-shutter mode, the VB/VD1940 is reported to produce a 100-dB full-resolution color image.

ST said the “on-chip Bayerization processing enables the user to reshuffle the color pixels of the RGB NIR 4X4 pattern into RGGB format,” which is compatible with a variety of SoCs.

The embedded processor manages cybersecurity features to ensure privacy in connected-vehicle applications, including mutual authentication and pairing of camera and electronic control unit (ECU), as well as video-stream authentication.

The hybrid sensor is available in bare wafers (VDB1940) and packaged in BGAs (VB1940). Samples are available with mass production planned for 2024 vehicles. The sensor is qualified to AEC-Q100 and is ISO 26262 compliant for use in functional-safety systems up to ASIL-B.