This Smart Vision Sensor Instantly Learns What to Detect

Pixy is a powerful vision sensor that can be reconfigured on the fly to identify almost any object. Teaching it what to find is as easy as holding the designated object in front of a camera while pressing a button for a few seconds. Using a color hue detection algorithm and powerful CPU, the device captures images at 50 frames per second as it searches for the designated object(s). Pixy can be used with or without a microcontroller, or as a combination of multiple units ― It’s perfect for robotics projects. Multiple Pixy units will grant robots complete 360 sensing, while a Pixy without a microcontroller can trigger events, switches, or servos.

The two primary drawbacks of image sensors are that they output tons of data — dozens of megabytes per second — and as result require a powerful processor to handle all that data, using up processing power that could otherwise be used for other tasks. Pixy overcomes these challenges through the inclusion of the powerful NXP LPC4330, 204-MHz dual-core processor, which only sends the most pertinent image information to the microcontroller, for example, the x and y coordinates of the detected object.

How does Pixy detect?
Pixy detects object placement through the use of a hue-based filtering algorithm that enables the extraction of precise hue and saturation data from each RGB pixel detected by the image sensor. Unlike most color-sensing algorithms, which are disrupted by changes in lighting and exposure, Pixy’s remains largely unchanged.

Pixy can remember up to seven simultaneously color signatures, allowing it to differentiate between seven different item types. Nevertheless, Pixy can sense hundreds of objects simultaneously provided there are no more than seven types. Using a connected compound algorithm, it differentiates the borders between each item. Next, the data is used to compile the size and location of each object type and reports it using one of its many interfaces.

Above: Pixy’s proprietary software in action sensing multiple objects of the same type

So how fast is Pixy?
Pixy sees everything; with a recording speed of 50 frames per second, Pixy snaps 50 image clips per second at 640 x 400 resolution or one image every 1/20th of a second. This allows Pixy to even track the trajectory of a bouncing ball covering 1 foot of distance every 20 ms.

Teach Pixy to track specifics
Educating Pixy to sense new objects is as easy as holding the designated object in front of the sensor and pressing a button until the RFD LED under the lens changes color to match the color of the object under its sensor. Pixy then produces and stores a statistical model of the colors in object it is searching for. Recall that up to seven tracking parameters can be remembered at any one point. Updating this list requires overwriting previous data.

Color code for extra tracking
Color tags, made from a combination of two or more colors, can be used to expand Pixy’s tracking capabilities beyond the initial seven signatures. Creating a recognizable pattern allows Pixy to identify any additional objects labeled with a color code and expand its purpose. Moreover, extra detection criteria will only improve Pixy’s accuracy.

For example, the color tags are an excellent way to help a robot navigate around an enclosed area without the use of electromagnetic waves. Place unique tags to identify doorways and charging stations and trigger specific actions upon detecting the patterns.

Technical specs
• Processor: NXP LPC4330, 204 MHz, dual core
• Image sensor: Omnivision OV9715, 1/4″, 1280×800
• Lens field-of-view: 75 degrees horizontal, 47 degrees vertical
• Lens type: standard M12 (several different types available)
• Power consumption: 140 mA typical
• Power input: USB input (5 V) or unregulated input (6 to 10 V)
• RAM: 264 Kbytes
• Flash: 1 Mbytes
• Available data outputs: UART serial, SPI, I2C, USB, digital, analog
• Dimensions: 2.1 x 1.75 x 1.4 inches

Visit Pixy's Kickstarter page to learn more about the sensor or to procure your own.