© Sick



Image processing increases quality, efficiency and safety in production in a sustainable manner. Practical applications illustrate the latest technical developments, trends and visions of the future of industrial image processing, also known as machine vision.

By Nikolaus Fecht

© Allied VisionEmbedded systems are becoming more popular in image processing due to the rapidly increasing performance of embedded processors. This trend is also reflected in the automation of manufacturing processes. When it comes to image processing, currently available embedded systems are reaching their limits, since they are based on simple camera modules whose images are processed almost entirely on a host processor. In traditional PC-based image processing (machine vision), on the other hand, the camera already takes on a significant part of image optimization.

How can powerful machine vision cameras be combined with the advantages of an embedded system? One approach would be to model embedded camera modules on machine vision cameras and equip them with an image processor. Instead of a programmable logic device (field-programmable gate array - FPGA), which would be too expensive for embedded technology, a system on a chip (SoC) is used. For Paul Maria Zalewski, a technical expert at Allied Vision Technologies GmbH from Stadtroda, this is a paradigm shift for embedded vision systems. Similar to PC-based image processing, the camera corrects and optimizes the images, while the central processor is merely used for application-specific image analysis.

Using processing power

Machine builders are increasingly using embedded systems in automation. "All components are connected and increasingly integrated into an overall system," explains the expert. Embedded-ready cameras meet the industrial requirements for hardware and software, performing image processing tasks that machine vision systems have completed until now. This makes it easier for the mechanical engineers to migrate from PC-based to embedded systems. "The products which are based on this technology are currently being tested internally and externally. The start of series production is scheduled for 2018," says Zalewski.

Machine vision has been key to in-process quality control for a long time. Bi-Ber GmbH & Co. Engineering KG in Berlin have recently developed a 3D surface inspection system for controlling the quality of molds used in food production. "Plastic molds for chocolate, for instance, are subject to high wear and tear due to thermal and mechanical stress," explains Ronald Krzywinski, Managing Director of Bi-Ber. "In order to prevent plastic flakes from ending up in the product, the molds must be constantly checked for defects." Until now, the fast systems for three-dimensional surface inspection have only inspected the easily-breakable root faces on the underside of the mold, but not deeper areas.

Doubling of the field of vision to improve measurement processes

The solution to this problem are 3D profile scans with reflex optics. "In the reflex process, the camera's field of vision passes over a semi-transparent mirror which splits the ray of light into two separate rays which overlap from the view of the camera," explains designer and simulation specialist Sascha Kühl. "The first ray path passes through the semi-transparent mirror, and is reflected in order to increase the accuracy during the scattering on two optical mirrors, hitting the mold at an angle of 45 degrees. This change also enables a significant reduction of the dimensions of the measuring systems." The second ray path is reflected by the semi-transparent mirror and directed to the other side of the laser. After the first reflection, the ray is reflected twice more by optical mirrors. "Since the second ray path is reflected one additional time, has the same working distance and points at the opposite side at a 45° angle onto the laser line, the orientation and scaling of both ray paths are identical," says Kühl.

Andreas Behrens © SickAnd because both perspectives are exactly the same, yet come from opposite directions, the camera sees the lines overlapped. If one of the paths is concealed, the laser line will be darker. This reflex method provides operators with much more leeway in adjusting the shutter speed and other settings on the mold color than other procedures. According to Bi-Ber, it is ideal for fast production processes with an average recording speed of 2.5 million pixels per second. "Since a reflex system increases the accuracy, practically shadow-free and precise 3D images of samples with complex geometries can be captured at high frequency," Krzywinski adds.

Industrie 4.0 presents both opportunities and challenges

"The fourth industrial revolution presents both opportunities and challenges for visionary perspectives," says Andreas Behrens, Head of Marketing & Sales Barcode RFID Vision at Sick AG from Reute. "Fundamental changes are necessary. Intelligent, seeing sensors capture a magnitude of data and are more than mere systems for controlling industrial manufacturing processes." Technologies like machine vision are helpful in this context, as they create the leeway for new solutions. The data recorded by the sensor in the inspection process can be used to take concrete measures to avoid manufacturing rejects. Vision sensors which capture 1D barcodes and 2D codes are also a hot topic for Industrie 4.0. These can be used, for example, for solving sorting processes more reliably.

When apps come into play...

A new system developed by the Baden-based company, which enables users to integrate their own ideas and work with individual apps, serves as the technological enabler for these types of innovations. The system uses programmable sensors and a powerful multi-camera and sensor processor for cross-technology image processing, sensor fusion and data collection. Programs for the development, implementation and management of apps serve as software. The Sick AppSpace Developers Club, in which developers from Sick and their customers share information and define further development steps of the system, serves as a great platform for exchanging experience. The platform enables app programmers to design user-friendly solutions with lean user interfaces.

Some users would like to be able to customize their image processing systems to their specific application themselves without the support of the product manufacturer. A solution for this are cameras whose FPGA logic block (for experts: field-programmable gate array) can be programmed. This is how huge volumes of image data can be processed in real time with the help of special algorithms.

Self-programmed image processing systems

The cameras of Baumer Optronic GmbH from Radeberg - the Vision Competence Center of the Baumer Group - enable users with the respective software know-how to directly integrate application-specific image processing tasks into the FPGA - without knowledge of special, complex programming. "This is how high-resolution image data can be processed quickly and cost-efficiently. In addition, performance can be increased and the system configuration simplified-– not to mention the reduction in system costs," says product manager Mirko Benz.

Mirko Benz © BaumerAccording to the product manager, the often complex and time-consuming process of programming the FPGA is the biggest challenge of application-oriented image processing in the camera. Tried-and-tested approaches with hardware description language make the development tedious and expensive. That's why Baumer has seized upon a new approach. "Thanks to a graphic programming environment, users without in-depth knowledge of programming can implement complex, application-specific algorithms quickly, flexibly and tailored to their applications," emphasizes Benz. "This simplifies the design process, reduces costs, accelerates the development and protects knowledge."

The use of complex FPGA filters enables a reduction in the amount of data to be transferred and analyzed. The load of PC-based machine vision systems caused by CPU-intensive algorithms can thus be relieved, reducing the demand on processing power. A nice benefit for users is that they do not need to invest as much money in powerful hardware. Thanks to the flexible programming of FPGAs in cross-sector applications, the cameras can be used for processing image data.

The twin concept of Matrix Vision GmbH from Oppenweiler allows for a simple introduction into machine vision without programming and expert knowledge. "We have used our expert knowledge to develop a camera software that enables newcomers to the world of industrial image processing to solve a multitude of typical tasks on their own," explains Managing Director Uwe Furtner.

Automatic selection of algorithms

According to Matrix Vision, the intuitive twin concept requires neither knowledge of image processing nor programming. "This is thanks to the integrated digital image processing system, which automatically analyzes the images captured," says Furtner. "A special program independently chooses suitable algorithms and, if necessary, filters, and parameterizes them correctly." In addition, operators can make use of a manageable number of tools with exact descriptions of their function, such as "Find object". The system thus speaks the language of the users, who are well able to describe what they want to achieve, but typically lack in-depth knowledge of machine vision. The tools are intended to help users accomplish visual inspection tasks quickly and cost efficiently in a matter of minutes.

Uwe Furtner © Matrix VisionWhat is more, the twin concept is also set to make the digital transformation easier. "In light of Industrie 4.0 and the digitalization of production systems, many companies are currently facing huge challenges," says Furtner. "However, the rapidly increasing demand on machine vision solutions cannot be met due to limited manpower and a lack of specialists." Furthermore, some companies are simply not able to develop their own machine vision competencies due to the complex programming processes, the substantial effort in terms of implementation and usability, and the required expert knowledge.

At the "Schweißen & Schneiden" trade fair in Düsseldorf in August, Vitronic Dr.-Ing. Stein Bildverarbeitungssysteme GmbH from Wiesbaden demonstrated how the value-adding quality testing of weld seams can be improved by means of automatic reworking directly in the inspection cell using intelligent image processing. The solution presented at the trade fair had been created for an automotive supplier. "In industrial manufacturing, quality tests do not count as value-adding processes," explains Markus Maurer, Director Sales Industrial Automation at Vitronic. "They reduce the risks of faulty components, but can also reduce the output. We are expanding the process of quality testing by creating added value." According to Maurer, the use of detailed data from optical quality analyses, newly developed software algorithms and classifications will facilitate the automatic reworking of soldered and weld seams directly in the inspection cell. This process optimization ensures immediate value creation - by reducing reject rates in case of faults.

No more interruptions

Increased manufacturing efficiency is the most important advantage of this solution according to Maurer. "Production processes will no longer be interrupted due to manual processes," says the expert. "Consistent automation ensures precise, constant and reproducible results in the welding of axle components, car bodies, wheels, and battery modules." Furthermore, operators can use the data collected to improve processes or reduce reject rates.

This form of quality assurance also has positive effects on Industrie 4.0 solutions and the increasing networking of production processes and plants. Here, inspection systems are needed that can supply and analyze the data, and feed it into the related processes. "By combining various inspection processes and results, we are able to optimize the efficiency and quality of processes," says Maurer. "In addition, storing the data in a central database also means improved traceability and transparency."

Wolfgang Mahanty © OptimumWolfgang Mahanty, Managing Partner of the Karlsruhe-based Optimum datamanagement solutions GmbH even goes one step further. In his view, intelligent image processing makes it possible to create a digital shadow for manual manufacturing processes. "The digital shadow is generated by means of intelligent image processing while a touch monitor leads an employee through the process," says Mahanty. In a subsequent step, it is checked whether the article was installed using the right quantities and position. According to Mahanty, the difference compared to other assistance systems is that the properties are inspected directly on the article or component. Other assistance systems only check whether the employee has picked up the right component from the right tray.

The solution developed by the Karlsruhe-based company stands out thanks to the fact that the intelligent machine vision unit is connected to a server-based, higher-level host computer (ERP, MES or PIS), on which the digital shadow is stored. The image data is processed in an intelligent data processing environment that combines database technology and image processing capabilities. This means that the system does not rely on the typical machine and plant control systems (programmable logic controllers - PLC) used in conventional machinery environments. Instead of merely mapping signal outputs, the system is thus able to map entire processes.

Bridging the gap to Industrie 4.0

According to the company, the combined system is characterized by high precision and increases efficiency by around 20 percent. Furthermore, the automated system relieves the mental strain on operators. "The result is a permanent comparison of target and actual process data, making it possible to identify potential sources of errors early on and avoid them," says Mahanty. "At the same time, employees benefit from an overview of how the process actually works, preventing failures from the get-go." Above all else, Mahanty places importance on the fact that it is not about controlling, but supporting the employees. The system masters even highly complex processes and visualizes them for employees, so that they can concentrate 100 percent on their activities. "Our system is particularly suitable for areas such as incoming goods inspection, assembly and final inspection, as it helps to turn the vision of Industrie 4.0 into reality," summarizes Mahanty.

Further Information

VDMA Machine Vision   |   Allied Vision Technologies   |   Baumer Optronic   |   Bi-Ber   |   Matrix Vision   |   Optimum datamanagement solutions   |   Sick   |   Vitronic Dr.-Ing. Stein Bildverarbeitungssysteme 

Anne Wendel, VDMA Robotics + Automation, Machine Vision.