Tokyo University of Science
Tokyo University of Science Tokyo University of Science

Key Points

Overcoming Optical Barriers: The "HEAPGrasp" technology enables robotic systems to identify and grip transparent and reflective objects, which are challenging for conventional 3D sensing approaches.

Operational Efficiency: The system achieved a 96.0% success rate under test conditions, outperforming baseline methods in both accuracy and speed.

Throughput Optimization: An optimized camera path reduced overall execution time by 19% and camera travel distance by 52%, supporting improvements in industrial cycle time ("takt time").

Strategic Industrial Impact: The approach targets automation bottlenecks in manufacturing and logistics, particularly in handling transparent trays and glossy packaging.

Versatile Robotic Manipulation
Versatile Robotic Manipulation: Demonstration of the HEAPGrasp system successfully identifying and gripping a transparent container (left), an object encased in a transparent plastic bag (center), and a highly reflective glossy item (right).

Researchers at the Tokyo University of Science have reported progress in industrial automation with the development of a robotic perception system capable of high-speed, precise manipulation of transparent and reflective objects. Led by Associate Professor Shogo Arai and Ginga Kennis, the team introduced "HEAPGrasp" (Hand-Eye Active Perception to Grasp), a methodology designed to address a long-standing challenge in smart manufacturing.

Conventional robotic systems rely on depth sensors that often struggle when encountering surfaces that transmit or reflect light, such as glass containers or metallic packaging. The HEAPGrasp system addresses these limitations by utilizing a single RGB camera and integrating semantic segmentation with a "Shape from Silhouette" approach. This enables the reconstruction of 3D geometries from multiple visual contours without requiring specialized sensor arrays.

Comparative Performance Analysis
Comparative Performance Analysis: A benchmarking of grasping success rates and operational efficiency between existing baseline methods (VGN, GraspNeRF) and the newly developed HEAPGrasp frameworks.

A key contribution of this research is the implementation of a cost function that optimizes the camera's trajectory. In industrial environments, cycle time—or "takt time"—is a critical metric for profitability. While multi-view observation can improve accuracy, it may also introduce delays due to camera movement. The team demonstrated a balance between these factors, reducing camera travel distance by 52% and overall execution time by 19% compared to conventional multi-view methods.

In benchmarking against existing frameworks such as GraspNeRF and VGN, HEAPGrasp demonstrated improved robustness. While traditional methods showed reduced success rates of 52–68% for transparent and glossy objects, the proposed system achieved a success rate of 96.0% under the tested conditions. In addition, it showed the ability to handle objects not included in the training phase.

"In manufacturing and logistics, transparent trays and glossy bags remain bottlenecks that often require manual handling," said Associate Professor Shogo Arai. "Our aim was to develop a mechanism that can 'see and grasp' with minimal movement, supporting the automation of processes that have traditionally depended on human labor."

The findings suggest potential applications for the global logistics and manufacturing sectors. By reducing the need for manual sorting and pre-adjustment of equipment, HEAPGrasp could support automation in high-mix, low-volume production environments. The research has been published in IEEE Robotics and Automation Letters and is scheduled for presentation at the 2026 IEEE International Conference on Robotics & Automation (ICRA).

Topics AI, Automation