This painting robot is controlled by a person’s eye movement

It’s no Pablo Picasso, but a brush-wielding robotic arm that can be controlled by a pair of eyes has just painted its first picture: a simple sketch of a house, the sun, and some grass.

Developed by Aldo Faisal of Imperial College London and his team of engineers, the system takes a different approach to that used for other robot artists, which are typically programmed to draw. Instead of operating solo, Faisal-and-team’s bot extends the human body by providing it with a third, artificial arm. According to Faisal, it’s the first step to human augmentation with additional limbs. 

The robotic arm's masterpiece is considered the first step to human augmentation with additional limbs.

Having an extra arm or two sure sounds handy, and from a robotics standpoint, a mechanical arm that performs tasks is indeed possible, but the real challenge lies in how this limb might be controlled so that it feels natural enough to be of benefit. Faisal and his engineers have developed computer software they call Gaze Space technology, which is a system capable of tracking a person’s eye movements and converts them into robotic commands. The reason behind this approach is that where we’re looking can give a useful indication of what we intend to do with our hands.

Graduate student Sabine Dziemian put the system through its paces. She described controlling the arm to be “intuitive” and was able to paint lines by looking first at the start point and then at the end point. In order to change colors, she had to blink three times, which saw the tip of the brush poised over three different options. From where she looked, she could dip the brush into the paint color of choice. Three blinks returned the brush to the canvas.

Watch the video below to see the artsy robotic arm assist in the creation of a masterpiece.

Controlling robots like this may prove useful in other ways in the future, such as assisting with motor impairment or other disabilities. It’s for this reason that translating natural functions of the human body, specifically thought, into robotic commands is being explored.