Fraunhofer IOF at Automatica: Measurement of transparent uncooperative surfaces

What used to take 15 seconds now takes less than two: thanks to new single-shot technology, the goRrobot3D system, developed by researchers at the Fraunhofer Institute for Applied Optics and Precision Engineering IOF, will be able to capture three-dimensional objects even more efficiently in future - even if they are transparent or black. The institute will be presenting the technology for the first time from June 24 to 27 at the Automatica trade fair in Munich in Hall 4, Stand 319. What is clearly visible to the human eye poses a real challenge for robotic systems: transparent glass or plastic, reflective metals or deep black surfaces. So-called "uncooperative" surfaces are difficult for conventional sensor systems to detect. Researchers at Fraunhofer IOF are addressing this challenge with the goRobot3D system, which uses intelligent thermal imaging to detect such surfaces. Researchers at Fraunhofer IOF have been working on thermal 3D sensor technology for several years. They have now significantly enhanced this technology and achieved efficient optimization for automated object detection. The measurement and evaluation time of transparent or deep black objects by goRobot3D has been reduced from 15 to just under 1.5 seconds. To this end, a new projection method for thermal 3D sensor technology was developed at the institute.

Single-shot technology instead of serial recording

The method developed transfers single-shot technology to thermal 3D measurement technology. "With our method, the surface of the measurement scene is heated in a structured manner. A thermally statistical dot pattern is emitted on the surface of the objects and recorded using two thermal imaging cameras. Using spatial cross-correlation, a 3D result can be obtained from the recorded image pair," explains Dr. Martin Landmann, a scientist in the Department of Imaging and Sensor Technology at Fraunhofer IOF. Instead of using fringe projections to generate the pattern as before, two diffractive optical elements (DOEs) now generate an irregular dot pattern. Such DOEs use the principle of light diffraction to multiply the incident laser beam and divide it into a pattern. By cleverly combining the DOEs, the researchers were able to project the required dot pattern onto the transparent object efficiently and within a very short time for the first time. Previously, a large number of image pairs had to be captured and evaluated in order to process the transparent materials - a time-consuming process. "Instead of capturing several hundred pairs of thermal images, as was the case with the previous method, our newly developed method can reconstruct the 3D information with just a single pair of images within a few milliseconds. This reduces the overall measurement and evaluation time by an order of magnitude," says Dr. Landmann. The recorded 3D data is analyzed using artificial intelligence. Suitable gripping points and directions are derived and transmitted to a robot arm with a suction gripper. "We use a 'bin-picking' method for this," explains the Fraunhofer researcher. "You could say a 'bin-picking' - in other words, the targeted gripping of objects from chaotic environments."

Production processes without cycle interruptions

The drastically reduced recording and evaluation time creates new possibilities for automated industrial processes, for example in production facilities or product design. Robots can not only reliably identify and grip transparent or dark objects, but also continue working with virtually no interruption to the cycle. "While one object is being handled, the next measurement can already be taking place. This creates flowing production processes," emphasizes Martin Landmann. Thanks to its modular structure, the single-shot technology can be flexibly integrated into a wide variety of applications.