Modern production and logistics environments are racing toward greater automation and flexibility. One of the biggest remaining challenges is teaching robots to pick up randomly placed parts from bins — a task humans do instinctively but machines find extremely difficult. This process, known as bin picking, demands robots that can handle unstructured, unpredictable situations.
Parts in a bin are often jumbled together, overlapping, or partially hidden. Traditional robots thrive in repetitive, well-ordered settings, not in this kind of visual chaos. But thanks to rapid advances in 3D vision technology, image processing, and artificial intelligence (AI), robotic bin picking is becoming a practical reality. Today’s robots can identify individual objects, determine their exact position and orientation, and grasp them accurately using the right grippers. Tasks that once required human hands, such as feeding parts to assembly lines or sorting goods in warehouses, can now be automated.
The demand for these systems is growing fast. Industries face pressure to increase efficiency, maintain quality, and reduce costs, all while coping with shortages of skilled technical workers. By automating repetitive, physically demanding jobs, companies can redeploy employees to more creative, value-added work. In short, bin picking is a key step toward flexible, fully automated production systems that can adapt to modern, high-mix manufacturing.
Current state of affairs
Many standard bin picking solutions still require fine-tuning for each specific product. The robot must recognise the correct object, decide which one to grab first, and calculate the best grip point. These steps depend heavily on the object’s shape, material, and the type of gripper used. When bins contain mixed or irregular objects, even advanced systems often struggle. This is where collaborative research becomes essential.
Research within Tech For Future
To address these challenges, three Dutch companies within the TValley innovation cluster in the eastern Netherlands — Riwo, Viro, and Voortman Steel Machinery — teamed up with Saxion University of Applied Sciences’ Smart Mechatronics and Robotics group. Together they launched an applied research project within the Tech For Future programme.
TValley brings together companies, researchers, and students in robotics, AI, and mechatronics. The goal is to develop practical, market-ready innovations through shared knowledge and collaboration.
Tech For Future is Saxion and Windesheim’s Centre of Expertise in the field of high-tech systems and materials. Within this programme, research groups conduct practice-oriented research in public-private partnerships, with a focus on co-creation and joint knowledge development.
The Robbin Project focused on use cases that lack suitable commercial solutions. Two applications were explored in depth and developed into a modular demonstrator:
PLASMA-CUT STEEL PLATES
Each plate has a unique shape and may contain holes or cut-outs. In addition to bin picking, the system has to identify the plate (by matching CAD data), detect which ones are free to pick up, and find a secure gripping point, ideally near the centre of mass and away from any holes. Important challenges include determining which plates are free (i.e. not stuck under other plates ) and finding a secure gripping position, ideally close to the centre of mass and not in or near a hole/cut-out.
IRREGULARLY SHAPED OBJECTS
Here the challenge was to determine the object’s pose (its position and orientation) of the objects and calculate how the robot should pick up objects reliably.
The following research questions were central to the project:
What is the current state-of-the-art in conventional computer vision algorithms, and what possibilities do recent AI solutions, such as deep learning, offer?
Which gripper and sensor technologies are most suitable for the applications of the industrial project partners?
Development of the demonstrator
A modular demonstrator was developed to test various solutions. The setup can easily switch between different use cases, robot brands and types, sensors (including Zivid 2+, Wenglor MLBS, Zed 2, and Framos), and gripper technologies such as vacuum, magnetic, and two-finger grippers.
Among other components, a Zivid 2+ structured light sensor, made available by the Fraunhofer Innovation Platform for Advanced Manufacturing, was used for the system. Thanks to its modular software architecture, the demonstrator allows results to be reused and extended in future projects involving bin picking and product handling.
For the conventional approach, a data pipeline was built around the opensource library Open3D. The results proved promising, bringing industrial application within reach. In the case of the steel plates, the system successfully distinguished between different plates regardless of shape, identified which ones were on top and therefore suitable for gripping, and calculated the optimal magnetic grip point — even for plates with holes in the centre.
The performance was both accurate and robust. However, developing this solution required substantial application-specific engineering. This challenge prompted the research team to explore what modern AI-based techniques could contribute.
Application of AI-based solutions to bin picking
To determine which recent AI algorithms are best suited for bin picking, the research team carried out a feasibility study on deep learning frameworks focused on object segmentation and pose estimation. Two promising candidates were identified: SAM-6D, based on Meta’s Segment Anything model, and FoundationPose, developed by NVIDIA Labs. Unlike many common deep learning solutions, these models can recognise and orient objects with little or no task-specific training data (i.e. zero/one/few shot training).
Based on feasibility analysis, the team integrated FoundationPose into the demonstrator. Using only a few reference images or CAD models, the algorithm successfully guided the robot to pick and move various objects with different grippers.
The results were encouraging, showing that AI-based vision systems can significantly enhance flexibility in bin picking. However, further optimisation is still needed before the technology can meet industrial standards for robustness.
Conclusion and follow-up
The project has provided the partners with valuable insight into the possibilities and limitations of both conventional and AI-based bin picking solutions. The knowledge now serves as a foundation for developing smarter, more flexible gripping systems for the manufacturing industry.
The Smart Mechatronics and Robotics research group continues to refine these solutions. In future projects, they aim to build more robust and widely applicable solutions that contribute to fully automated, adaptive production environments.
