A Finnish bread company is reaping the benefits of adopting a robotic system that includes a high-performance vision system using Matrox software.
The team at Finnish company Kine Robotics was asked to develop a conveyor tracking system with on-the-fly robot picking for bags of bread loaves and rolls for an industrial bakery. Product sizes are irregular, and shapes differ from one type of bread to another, making it a challenge to determine the location and orientation of the bags on the conveyor.
The transparent plastics packaging also makes it difficult to detect with optical sensors, due to low contrast and partial reflections.
Prior to the upgrade to a high-performance vision system, the process involved manual removal and depositing of bread packs, which was potentially error prone and inefficient.
The installation has resulted in a lower error-rate than the prior manual handling and with a higher throughput of goods. With the new vision system, the robot grip can handle between 25 and 30 packs per minute. “We’re already considering the possibility of adding some measurement limits for the products to ignore or flag faulty products as they move through the system,” says Kine’s project manager, Tero Urponen.
The team at Kine knew that machine vision was the only possible solution for this type of system. The company sought to develop the vision system around Matrox Design Assistant X software and a Basler Blaze 101 3D Time-of-Flight (ToF) camera as the sensor. The ToF camera was selected as it eliminated the need for separate lighting, and reduced concerns about contrast and reflections.
“We really liked how smoothly the 3D ToF camera integrated with Matrox Design Assistant X software,” confirms Urponen. “In other projects, we have had multi-camera systems that can get a bit cumbersome, whereas in this case, it was very nice to have one system to control everything instead of having to program each camera separately.”
For this food packaging installation, the products being examined posed some serious challenges for the vision system: different types of breads have unique shapes, sizes, and colours, and the semi-transparent packaging can obscure the contents, making it harder for the system to distinguish between products.
The inspection system affords real-time measurement and provides precise 3D data to the vision software. The camera is triggered from a photodetector by way of a B&R programmable logic controller (PLC) communicating with the vision program via a user datagram protocol (UDP). The PLC also synchronises the robot by similar means.
An industrial panel PC from B&R serves as the platform to run the Matrox Design Assistant X runtime environment and the associated vision program. Handling the picking and placing is a Stäubli TS2-60 SCARA robot, which communicates via transmission control protocol (TCP) with a Stäubli VALtrack software library for conveyor tracking and picking.
The robot needed to be able to get a good grip on the plastics bags without causing any damage to the delicate bread or its packaging.
Camera data analysis is implemented using Matrox Design Assistant X software, which also produces the central interface to the vision system. The 3D ToF camera records the surfaces as a point cloud with over 300,000 XYZ coordinates; Matrox Design Assistant X converts the point cloud into a depth map, which is analysed using 2D vision tools to determine the grip points for the robot. The 2D tools provided by Matrox Design Assistant X work particularly well on depth-map data and allowed the team at Kine to effectively solve its application needs.
Expressly designed to require no operator interaction, the vision system is located directly above the incoming product conveyor, just before the robot working area, and functions completely autonomously.
Through use of the ToF camera and vision software, Kine saw a reduction in the overall costs related to machine vision by 75%, compared to traditional camera and lighting options. “Typically, there is a lot of trial and error with lights and camera settings to get proper contrasts for object detection, but the ToF camera requires no external lights, and the 3D information it supplies to the IDE is superior to any traditional colour/contrast-based detection technology,” says Urponen.