This article is part of an AspenCore Special Project on vision-guided robots. With the cost and size of cameras dropping and the power of image-processing software increasing, robotic systems are seeing a surge in new vision-guided applications, especially those using 3D. This Special Project explores hardware, software, and business issues surrounding this technology through a collection of interrelated articles, listed at the end of this one.
A robot’s ability to grasp and manipulate objects quickly and precisely makes them perfect for executing repetitive, dangerous, or exhausting tasks such as parts inspection, operating in fog, fire, or under water, or welding. As sensors that collect and analyze information, cameras are essential for the robots to navigate the landscape and avoid collisions with nearby objects. The different types of camera technology include 2D imaging, 3D sensing, ultrasonic, and infrared.
Types of technologies behind robot-vision cameras
2D-imaging technology
For a robot with machine vision that does not require information on depth or distance, conventional 2D digital cameras are the most popular choice. While digital cameras look like film cameras, they are based on entirely different scientific principles. Unlike a television, which projects millions of tiny colored light dots, or pixels, to make up images, a digital camera captures the light particles (photons) that bounce off of an object and turns them into electric signals, or pixels, that can be stored as numbers. A digital photo is a long string of numbers that describes each pixel. On screen, the long string of numbers is translated into pixels that make up an image. There are two types of 2D digital cameras: charged-coupled device, or CCD, and complementary metal-oxide semiconductor, or CMOS. CCD cameras offer higher resolution and lower noise than CMOS cameras but are also more expensive and less energy-efficient. Here is an excellent guide on choosing 2D cameras. Additionally, recent innovations have added new capability to 2D cameras. For example, Pinnacle’s high-dynamic-range technology is able to help 2D cameras capture much better video and images.
Figure 1: The photos show two different images captured with different dynamic ranges. The one (right) with high dynamic range capability yielded much better results. (Image: Pinnacle Imaging Systems)
Advantages: 2D cameras are effective with object identification. They can clearly define the edge of a target and collect many details of the target in high resolution.
Disadvantages: 2D cameras are sensitive to the light conditions in the environment. They also identify a mirror image as a second object instead of an image. Lastly, image capture is a slow process that requires the camera and the object to be relatively still.
For scenarios that require the acquisition of 3D information, cameras with ultrasonic, infrared, or 3D-sensing technology may be more appropriate.
3D-sensing technology
3D sensing is a powerful tool for robot navigation as it provides information on the volume, shape, position, orientation, and the distance from an object. 3D data can be generated from different processes such as stereo vision, structured light, and laser triangulation. Stereo vision calculates depth using images from multiple 2D cameras in one location or one camera in different locations using the principle of triangulation.
Structured light is great for measuring the 3D shape of an object using projected light patterns and a 2D camera system. A great deal of information on the depth of an object can be gleaned from analyzing the observed striped patterns.
Laser triangulation uses a 2D camera and a laser. The laser projects a line onto the target zone, and the 2D camera captures the reflected laser beams. The contours of the object can be calculated from the bending of the laser beams across multiple photos. However, it is relatively slow and can only work on still images, which means that a process must be interrupted for data collection.
Advantages: 3D-sensing technologies are highly accurate in measuring objects at close range, have better resolution than ultrasonic and infrared, and are more reliable when the environment is cluttered. In particular, laser triangulation works on tricky surfaces or in situations of low ambient light, delivering precise data even for low contrast objects.
Disadvantages: 3D-sensing technologies cannot identify mirrors or glass because they will identify the reflection as a second object. Also, they are sensitive to light conditions and perform poorly when the environment is too monotonous and lacks features. In addition, they are slow and can only work on still images, which means that a process must be interrupted for data collection. Lastly, the cameras employed in 3D sensing are expensive, bulky, and hard to calibrate.
For more on 3D-imaging techniques, check out this EDN article on methods for 3D vision.
Ultrasonic technology
Also called sonar cameras, ultrasonic cameras measure the distance between the camera and an object by measuring the time lapse between the transmission and the reception of sound waves. Ultrasonic cameras can also detect other ultrasonic sensors or other ultrasonic sensor-bearing robots nearby.
Advantages: Ultrasonic cameras have longer and wider range than other types of sensors. They will not be affected by light conditions (be it too much or not enough) or air particles such as dust, smoke, mist, vapor, and lint. Also, they are not hampered by the colors, transparency, or the reflective properties or surface texture of the object. Therefore, ultrasonic cameras are good for measuring liquid levels and distances, detecting objects, and avoiding collisions on land or under water.
Disadvantages: Ultrasonic cameras are poor in defining the edges of a target. Also, they work well with targets with a high density and reflective surface but not with low-density matters like foam or cloth. They are also affected by loud noises, temperature, humidity, and pressure. Lastly, they require a minimum sensing distance and cannot detect objects that are too close.
Infrared technology
Infrared sensors work by detecting the infrared (IR) rays emitted by an object. They can also calculate the distance or proximity of a target object by projecting IR light toward the object and receiving the reflected light.
Advantages: Infrared sensors can detect infrared light over a large area, operate in real time, and are cost-effective. They are great in defining the edge of an object and more helpful in identifying one object from another than ultrasonic sensors. For example, they help identify humans and other heated objects and distinguish them from reflective images (such as a mirror). In addition, IR cameras enable you to see in low-visibility conditions like smoke or fog. Like ultrasonic cameras, IR cameras can inspect the material underneath the surface and have seen application in the inspection of building insulation or leak detection.
Disadvantages: The IR sensor is very sensitive to IR light and sunlight and can only be used under low light. Also, infrared sensors do not work equally well on all materials because the materials transmit and refract IR light differently.
Choosing which type of camera to use
Each sensor type has characteristics that make it better for certain tasks and less so for others. For example, an ultrasonic sensor works fine for solid objects but not for soft or fuzzy objects. Also, they are not as good at distinguishing one object from another. On the other hand, while 3D sensing cameras offer great accuracy, they cannot work in darkness or underwater. Therefore, it is essential to compile the properties of your robot’s targets, the conditions in which your robot will operate, and the objectives of your robot’s tasks before beginning camera selection.
Camera mounting is another consideration: installing the camera near the end of the arm of the robot gives you a closer look at objects and better resolution, but this also means that the camera will move with the robot and the robot must stop while an image is taken. If you fix the position of the camera, you will be sacrificing resolution for the benefit of continuous imaging. Camera technology developers are integrating advanced technologies rapidly. For example, FLIR Systems’ FLIR Firefly camera family is able to integrate a camera and a vision processing unit (VPU) with added artificial intelligence ability.
Fig. 2: FLIR Systems’ FLIR Firefly camera family integrates a camera and a Movidius Myriad 2 vision processing unit (VPU) from Intel.
There is a myriad of cameras now available for vision-guided robots, with more coming along every day. To help you along the selection path, provided below are a flowchart that will help you identify the type of camera most appropriate for your needs and a chart of representative camera suppliers so that you can begin your search.
Flowchart for camera selection
Camera properties
Representative robotic-vision camera vendors
Check out the other articles in this Vision-Guided Robotics Special Project:
3D Vision Enhances Robot Opportunities — Vision-guided robotics (VGR) has long used 2D imaging, but the advent of cost-effective 3D is opening new application opportunities.
Open-source software meets broad needs of robot-vision developers — Robot-vision applications can bring a complex set of requirements, but open-source libraries are ready to provide solutions for nearly every need. Here are some of the many open-source packages that can help developers implement image processing capabilities for robotic systems.
3D vision gives robots guidance — Many options exist for 3D machine vision, each addressing different application needs.
Applications for Vision-Guided Robots — Perhaps the most significant recent developments regarding robotics have involved the combination of high-resolution imaging, artificial intelligence, and extreme processing capabilities.
Robotic vision electronics design for industry and space — Eighteen years into the new millennium, there are a number of exciting and evolving electronic innovations taking place. Among them is the development of “intelligent” robots for industry, especially in smart factories.
Advertisement
