Prophesee readies event-based vision sensors for smartphones

Extending its reach into smartphones and potentially wearables, Prophesee has announced a collaboration with Qualcomm Technologies, Inc. to optimize its event-based Metavision sensors with Qualcomm’s Snapdragon mobile platforms for next-generation smartphones. Bringing neuromorphic-enabled vision to mobile devices for improved image and video quality, the event-based vision sensors repair motion blur and other artefacts particularly in high dynamic scenes and low light conditions.

Initially targeting industrial applications to commercialize the event-based camera, Prophesee previously partnered with Sony to make a small, high-resolution event-based camera module, the IMX 636, using Sony’s 3D die stacking process. At the time, Prophesee noted the second market for the camera module is consumer technologies due to the shrink in size via the die-stacking process.

Under the multi-year collaboration with Qualcomm, the companies will enable native compatibility between Prophesee’s event-based Metavision sensors and software and Qualcomm’s Snapdragon mobile platforms. Mobile device developers will benefit from the improved camera performance, particularly in fast-moving dynamic scenes such as sports and in low-light, thanks to the event-based continuous and asynchronous pixel sensing approach, said the company.

Preliminary image deblurring results in a high-speed scene, using the Prophesee Metavision sensor and algorithms with the Snapdragon platform. (Source: Prophesee)

Qualcomm believes Prophesee’s technology is “game-changing” and will take “mobile photography to the next level.” This includes Prophesee’s achievements with event cameras’ shutter-free capability, which will improve the quality of photography on next-generation mobile devices powered by the Snapdragon platforms.

Prophesee explains that its sensors add a new sensing dimension to mobile photography by focusing only on changes in a scene, pixel by pixel, and continuously, at high speeds.

“Each pixel in the Metavision sensor embeds a logic core, enabling it to act as a neuron,” said the company. “They each activate themselves intelligently and asynchronously depending on the amount of photons they sense. A pixel activating itself is called an event. In essence, events are driven by the scene’s dynamics, not an arbitrary clock anymore, so the acquisition speed always matches the actual scene dynamics.”

The companies presented a live demonstration at Mobile World Congress (MWC) 2023 running on the Snapdragon 8 Gen 1 chip and event-based Metavision sensor. Prophesee development kits, designed to support the integration of the event-based vision sensors with devices that use the Snapdragon platforms, will be available this year.