Visual Servo Control in Robotics

Visual servo control integrates real-time visual feedback into robotic control loops, enabling adaptive and precise motion based on camera-derived information. Two principal strategies dominate the field: image-based visual servoing (IBVS), which uses raw image features—such as points, lines and moments—to generate control signals directly in the image plane, and position-based visual servoing (PBVS), which reconstructs three-dimensional pose to guide the robot through task-space trajectories. Hybrid schemes leverage both paradigms to maintain stability and convergence while mitigating issues such as feature loss, field-of-view constraints and modelling inaccuracies. Key challenges include accurate estimation of the interaction (image Jacobian) matrix, resilience to occlusion and illumination changes, and the computational demands of real-time image processing. Recent developments employ adaptive filtering and data-driven learning techniques to estimate control mappings online, reducing reliance on precise calibration. The global significance of visual servo control spans industrial automation, medical robotics, autonomous vehicles and aerial robotics, where precision, flexibility and responsiveness to dynamic scenes are paramount.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Contemporary studies have demonstrated the integration of advanced filtering techniques to enhance the accuracy and robustness of visual servo control. Dynamic visual tracking methods employ adaptive fading Kalman filters to adjust the weight of incoming image observations in real time, successfully coping with occlusions and uncertain camera parameters to maintain consistent trajectory tracking on manipulators. Uncalibrated systems have benefited from Bayesian approaches, such as unscented particle filters, for online estimation of the total image Jacobian matrix, enabling reliable control in the absence of explicit camera or robot calibration. Moreover, convolutional neural network architectures have been developed to learn direct mappings from image inputs to pose corrections in eye-to-hand configurations. These data-driven schemes leverage Siamese-inspired networks to encode desired and current views, regressing the relative pose to achieve high positioning accuracy without the need for explicit feature extraction or model-based interaction matrices.

Visual Servo Control in Robotics publication trend

The graph below shows the total number of articles in visual servo control in robotics across all publications each year (not limited to Nature Index journals).

Technical terms

Image-Based Visual Servoing (IBVS): A control strategy that directly utilises image feature errors to generate robot velocity commands.

Position-Based Visual Servoing (PBVS): An approach that reconstructs the three-dimensional pose of the target and uses it for control.

Image Jacobian Matrix: The interaction matrix relating changes in image features to robot motion.

Eye-in-Hand/Eye-to-Hand Configuration: Camera placement either mounted on the end-effector (eye-in-hand) or fixed in the workspace (eye-to-hand).

Kalman Filter: A recursive algorithm for optimal estimation of system states in the presence of noise and uncertainty.

Particle Filter: A Bayesian filtering method using weighted samples (particles) to estimate non-linear, non-Gaussian system states.

References

1. Dynamic Visual Tracking for Robot Manipulator Using Adaptive Fading Kalman Filter. IEEE Access (2020).
2. Image-Based Visual Servoing Control of Robot Manipulators Using Hybrid Algorithm With Feature Constraints. IEEE Access (2020).
3. Unscented Particle Filter for Online Total Image Jacobian Matrix Estimation in Robot Visual Servoing. IEEE Access (2019).
4. Convolutional Neural Network-Based Visual Servoing for Eye-to-Hand Manipulator. IEEE Access (2021).
5. A Comparison between Position‐Based and Image‐Based Dynamic Visual Servoings in the Control of a Translating Parallel Manipulator. Journal of Robotics (2012).