New ‘Robo Brain’ learns from the Internet — will provide robots with constantly updated knowledge

Researchers have developed a large-scale computation system called “Robo Brain” which will teach robots everything they need to know using publicly available Internet resources.

At present, the system is downloading and processing about 1 billion images, 120,000 YouTube videos, and 100 million how-to documents and appliance manuals.

The information Robo Brain gathers is translated and stored in a robot-friendly format. When the bot is hooked up to the system, it can draw on any of these resources in a split second for the purpose of performing a task.

“Our laptops and cell phones have access to all the information we want. If a robot encounters a situation it hasn't seen before it can query Robo Brain in the cloud,” explained Ashutosh Saxena, assistant professor of computer science at Cornell University, who worked with his colleagues at Cornell, Stanford, and Brown Universities, as well as researchers at the University of California (Berkeley) to develop this system. 

The group says robots will use Robo Brain to process objects in front of them and pick out related images within them so as to identify what the objects are; they’ll also be able to connect the object to images and videos from the Internet as well as related text. 

Here’s a good for-instance: Should a household robotic server come across a coffee mug, it can use Robo Brain to not only understand that the object is a coffee mug, but also that liquids can be poured in and out of it, that it can be picked up by the handle, and that when carried, if the mug is full, it should be carried upright, as opposed to when it is being taken out of the dishwasher or sink.

Robo Brain is able to employ such a comprehensive system of understanding courtesy of what computer scientists refer to as “structured deep learning.” What they mean by this is that information is being stored in many levels of abstraction (a lounge chair is part of the larger family of “chairs,” and chairs are part of the even larger grouping of “furniture” — a human sits on a lounge chair, but they can also sit on easy chairs, recliner chairs, bar stools, and more).

All of this information is stored in a form mathematicians refers to as a Markov model, which can be represented graphically as a set of points connected by lines referred to as nodes and edges.

Nodes represent objects, actions, or parts of an image, and each one is assigned a probability (that is, how much you can vary it yet still be correct—a table might be used to serve food, but the object can also be used to place things like grocery bags on it). While searching for knowledge on the object at hand, a robot brain makes its own connections and looks for one in the knowledge base that matches within those limits. 

“The Robo Brain will look like a gigantic, branching graph with abilities for multi-dimensional queries,” said Aditya Jami, a visiting researcher at Cornell, who designed the large-scale database for the brain.

The project was recently presented by Saxena at the 2014 Robotics: Science and Systems Conference. He has since launched a site for the project where you can learn more about the technology and how far along it is in terms of development: robobrain.me

Via Cornell