Visual Robot System Simulates Precision Loading for Oil Pipe Threading
In a significant advancement for automation in heavy industry, a team of researchers from Jiangsu University of Technology has developed a highly integrated virtual simulation system that enables precise, vision-guided robotic loading and unloading for oil casing threading machines. The study, led by Zhou Hao, Li Xingcheng, Liu Kailei, Xun Kangdi, and Liang Dong, demonstrates how combining 3D modeling, machine vision, and industrial robotics in a simulated environment can streamline complex manufacturing workflows, reduce operational risk, and enhance production efficiency—especially in sectors dealing with large, cylindrical components like petroleum pipelines.
The research, published in Mechanical Science and Technology for Aerospace Engineering, details the design and implementation of a dual-robot system capable of autonomously handling oil pipe joints—known as couplings—through a fully synchronized and intelligent process. Unlike traditional automation systems that rely on fixed positioning or manual teaching, this new approach integrates real-time visual feedback to dynamically adjust robot movements, ensuring accurate grasping and placement even when workpieces are slightly misaligned on conveyor lines.
At the heart of the innovation is a multi-software simulation framework that brings together Unigraphics (UG), ABB’s RobotStudio, and MVTec’s Halcon for image processing. This integration allows engineers to test and refine robotic workflows in a digital twin environment before deploying them on actual factory floors—a practice increasingly critical in modern smart manufacturing. By simulating the entire production line, including robot motion paths, sensor triggers, and inter-device communication, the team was able to identify potential bottlenecks, collision risks, and timing mismatches early in the design phase.
The system architecture revolves around two specialized ABB industrial robots: the IRB460, a high-speed four-axis robot used primarily for palletizing, and the IRB6640, a versatile six-axis robot suited for precision tasks. These robots operate in tandem with a CNC threading machine, dual conveyor belts for loading and unloading, a CCD camera for visual inspection, and a central electrical control cabinet that manages all input/output signals.
One of the key challenges in automating the handling of oil couplings lies in their size, weight, and variability in positioning. These cylindrical parts are often delivered on trays or conveyors in non-uniform orientations, making fixed-point robotic pickup unreliable. To overcome this, the team implemented a machine vision module using Halcon, a powerful industrial image analysis software. The camera captures real-time images of the workpiece, which are then processed through a series of algorithms to determine the exact center coordinates of the coupling.
The image processing pipeline begins with converting the raw RGB image into grayscale to simplify computation. Next, Gaussian filtering is applied to suppress noise—common in industrial environments due to lighting fluctuations or surface reflections. Following this, thresholding segments the image into foreground and background, isolating the object of interest. Connected component analysis then identifies distinct regions, and shape-based filtering is used to isolate circular features, which correspond to the cross-section of the pipe coupling.
Once the circular region is identified, the system calculates its centroid—the geometric center—which serves as the target pickup point for the robot. This coordinate data is then transmitted from the vision PC to the robot’s virtual controller via a TCP/IP socket connection, ensuring seamless integration between the perception and action layers of the system. The use of socket communication provides a stable and low-latency data channel, essential for maintaining synchronization in high-speed production cycles.
To make the system more user-friendly and accessible to plant operators, the researchers developed a custom human-machine interface (HMI) using C# and the .NET framework. This interface allows users to initiate image capture, monitor processing results, and visualize the detected center point in real time. Built upon Halcon’s code export functionality, the HMI integrates directly with the image processing engine, offering an intuitive dashboard that reduces the technical barrier for non-expert users.
The Robot 1, designated as the IRB460, is responsible for initial material handling. It retrieves raw couplings from a staging area and places them onto the loading conveyor. Once the part reaches the designated pickup station, it triggers a photoelectric sensor, signaling the Robot 2—the IRB6640—to take over. This second robot performs the critical task of loading the coupling into the CNC threading machine, where threads are precision-cut onto the ends of the pipe. After machining is complete, the same robot unloads the finished part and places it onto the discharge conveyor, while simultaneously loading the next raw piece—a clever dual-operation strategy that minimizes idle time and maximizes throughput.
To enable this level of coordination, the team designed custom end-effectors for each robot. For Robot 1, an internal expansion gripper was developed, allowing it to securely grasp the inner diameter of the coupling without interfering with adjacent parts or machinery. This inward-clamping mechanism is pneumatically driven, offering a balance of speed, reliability, and cost-effectiveness—important considerations in industrial automation.
Robot 2, however, faces a more complex challenge: it must perform both loading and unloading in rapid succession. To achieve this, the researchers engineered a dual-head gripper system, where two separate clamping units are mounted at an angle on a shared base. One head holds the next workpiece while the other retrieves the finished one. After unloading, the robot rotates its wrist to position the waiting raw part into the machine—effectively completing two operations in a single cycle. This design significantly reduces cycle time and enhances the overall efficiency of the production line.
The simulation environment in RobotStudio was used not only to model the physical layout but also to emulate the logical behavior of the entire system. Using Smart Components—a feature within RobotStudio that mimics PLC (Programmable Logic Controller) functionality—the team created dynamic models of sensors, actuators, conveyors, and grippers. These components were linked through a network of virtual I/O signals, replicating the real-world electrical and control logic that would govern the actual hardware.
For example, when the loading conveyor reaches its endpoint, a simulated digital input signal (di_dw11) is triggered, which in turn activates a sequence in the robot program. Similarly, output signals such as do_2jiaju00 and do_2zhuaqu0 control the activation and gripping action of the second robot’s tooling. By mapping these signals within the simulation, the researchers ensured that all devices would operate in perfect harmony, just as they would in a live production setting.
The control logic was implemented using ABB’s RAPID programming language, allowing for precise orchestration of robot movements and auxiliary functions. The main program for Robot 1 includes conditional checks for conveyor status, ensuring that it only performs loading or unloading when a part is actually present. Meanwhile, Robot 2 follows a more intricate sequence involving motion to multiple waypoints, waiting for sensor feedback, executing gripping commands, and coordinating with the CNC machine’s door mechanism.
One of the most notable aspects of the system is its flexibility. Unlike rigid automation systems that are difficult to reconfigure, this setup is designed to accommodate variations in workpiece size, position, and production rhythm. The vision system compensates for positional deviations, while the dual-gripper design allows for overlapping operations. This makes the system particularly suitable for medium-to-high volume manufacturing environments where adaptability is key.
To validate the system’s performance, the researchers conducted a 24-hour continuous simulation run. During this period, the virtual production line executed over 600 complete loading-unloading cycles without error. Timing analysis revealed that Robot 1 completed its individual tasks in approximately 16 seconds per cycle, while Robot 2 required 25 seconds due to the complexity of its dual loading/unloading function. The conveyor belts took 13 seconds to transport each part, and the CNC threading process itself lasted 65 seconds. The total cycle time for one full operation—including robot handling, transport, and machining—was measured at 132 seconds.
Critically, the simulation confirmed that all devices operated in perfect synchronization, with no collisions or timing conflicts. The robots moved along optimized paths that avoided obstacles, and the I/O signals triggered actions at precisely the right moments. This level of coordination would be extremely difficult to achieve through trial-and-error on a physical system, underscoring the value of virtual commissioning.
The implications of this research extend beyond the oil and gas industry. The methodology can be adapted for other cylindrical or tubular components used in aerospace, automotive, and construction sectors. For instance, similar systems could automate the handling of hydraulic cylinders, drive shafts, or boiler tubes—parts that share many of the same geometric and handling challenges as oil couplings.
Moreover, the integration of machine vision with robotic control represents a shift toward more intelligent and autonomous manufacturing systems. As industries move toward Industry 4.0 and smart factories, the ability to perceive, interpret, and respond to real-time conditions becomes increasingly important. This study demonstrates a practical implementation of such a system, bridging the gap between theoretical research and industrial application.
From a safety perspective, the simulation-based approach reduces the risk associated with testing complex robotic systems in live environments. Engineers can identify and correct potential hazards—such as robot collisions or timing errors—before any hardware is installed. This not only protects expensive equipment but also safeguards human operators who may be working nearby.
The economic benefits are equally compelling. By optimizing robot paths and minimizing cycle times, manufacturers can increase throughput without adding additional machinery or labor. The dual-robot, dual-gripper design also reduces the need for multiple workstations, lowering capital expenditure and floor space requirements.
Looking ahead, the research team suggests several avenues for future development. These include integrating force feedback for more delicate handling, expanding the vision system to detect surface defects or dimensional inaccuracies, and incorporating machine learning algorithms to improve positioning accuracy over time. Additionally, the system could be enhanced with wireless communication protocols or cloud-based monitoring for remote diagnostics and predictive maintenance.
In conclusion, the work by Zhou Hao, Li Xingcheng, Liu Kailei, Xun Kangdi, and Liang Dong presents a comprehensive and practical solution for automating the loading and unloading of oil pipe threading machines. By leveraging virtual simulation, machine vision, and advanced robotics, they have created a system that is not only efficient and reliable but also adaptable to changing production demands. Their findings offer valuable insights for engineers and manufacturers seeking to modernize their operations through intelligent automation.
The study was supported by the National Natural Science Foundation of China (Grant No. 51805228) and the Natural Science Foundation of Jiangsu Provincial Higher Education Institutions (Grant No. 18KJB460010), underscoring the importance of academic-industry collaboration in advancing manufacturing technology.
Zhou Hao, Li Xingcheng, Liu Kailei, Xun Kangdi, Liang Dong, Jiangsu University of Technology, Mechanical Science and Technology for Aerospace Engineering, DOI: 10.13433/j.cnki.1003-8728.20200279