Design and implementation of a 6-DoF robot arm control with object detection based on machine learning using mini microcontroller

Abstract

This research presents a novel approach to robotic manipulation by integrating an advanced machine learning-based object detection system on a resource-constrained AMB82-Mini microcontroller. Employing a lightweight, quantized YOLOv7-tiny model, the system achieves real-time object localization with high precision, enabling a 6-DoF robotic arm to perform complex pick-and-place tasks autonomously. The framework incorporates a machine learning-driven perception pipeline that interfaces with a kinematic solver to compute precise joint trajectories, enhanced by adaptive motion smoothing techniques. A closed-loop control system, augmented with sensor feedback, ensures robust performance across varying payloads. Experimental results validate the system’s efficacy, achieving consistent task success rates and computational efficiency on an embedded platform. This work demonstrates the potential of embedded machine learning to enable scalable, cost-effective automation solutions, offering insights into the synergy of perception and control in robotic systems.

Introduction

Robot arms are pivotal components in industrial automation, valued for their flexibility in object manipulation^1,2,3. The adoption of 6-degree-of-freedom (6-DoF) robotic arms has spurred advancements in nonlinear control strategies^4,5. This study focuses on designing and implementing a control system for a 6-DoF robotic arm using a novel algorithm that integrates object detection, microcontroller-based control, and a voltage source parallel strategy to ensure system stability. A 32-bit ARM Cortex microcontroller serves as the main processor for controlling the robotic system. Machine learning, specifically a convolutional neural network (CNN), is employed for real-time object detection. The vision system captures and processes images to determine object coordinates, which are then relayed to the microcontroller for precise actuator control. A voltage source parallel hysteresis modulation method is applied to manage the complex electromechanical behavior of the robot, ensuring stable and chatter-free operation.

The distinct contribution of this work lies in the integration of an object detection algorithm with an ARM Cortex microcontroller for cost-effective, intelligent control of a six-axis robot arm. Experimental validation on a physical system demonstrates high accuracy in object detection and tracking, with no creepage or control chattering. The system exhibits low root mean square error, rapid convergence, no overshoot, and stable steady-state performance. The growing demand for industrial robots in smart factories especially in automotive, electronics, and machinery sectors has heightened the need for accessible control solutions. Six-axis articulated robots are particularly valued for their flexibility and ease of use in point-to-point operations. However, their perceived complexity remains a barrier for non-expert users. This paper addresses this gap by proposing a low-cost development environment featuring trajectory planning, object detection, and machine learning for intuitive control. The robotic system is equipped with a full-HD CMOS camera on its end-effector to enable perception functions, including moving object detection. The microcontroller processes image data and executes control commands, allowing the arm to track and grasp moving objects. The research methodology includes simulation, functional testing for pick-and-place tasks, machine learning model design, and the development of a user-friendly control interface.

Unlike prior works where object detection and robot control were handled separately, this study unifies both functions within a single microcontroller framework. By combining autonomous learning and real-time image processing, the system offers a streamlined and efficient solution for industrial applications requiring precision and adaptability. The proposed approach enhances production accuracy and efficiency, providing a practical and user-friendly implementation of robotic automation supported by machine learning and embedded vision. Further, the paper’s contribution and innovation are summarized as follows:

• Develops and implements a unified control system on a single 32-bit ARM Cortex microcontroller that synergistically combines real-time machine learning-based object detection (using a CNN) with nonlinear actuator control for a 6-DoF robotic arm.

• Ensures robust performance and mitigates control chattering in the complex electromechanical system through a unique combination of real-time adaptive tuning and a voltage source parallel hysteresis modulation technique.

• Establishes a low-cost development framework that simplifies the control of advanced 6-DoF arms, making sophisticated automation more accessible to users without deep expertise in robotics, thereby addressing a significant barrier to adoption in SMEs.

• Moves beyond simulation to demonstrate the system’s efficacy on a physical robot, verifying high object detection accuracy, exceptional control stability (no overshoot, low RMSE, fast convergence), and reliable pick-and-place of moving objects.

• Unlike conventional approaches that treat vision and control as separate subsystems, this work innovates by fully integrating real-time image processing and robotic motion control within a single, low-power microcontroller unit.

The subsequent sections of this paper are structured as follows: Sect. 2 presents the literature review. Section 3 provides an explanation of the proposed materials and methods description. Section 4 presents a detailed examination of the results and discussion. Finally, Sect. 5 concludes the paper.

Literature review

Many robotic companies and research institutions have developed various arm robots using different controllers available today. In the following, some related literature will be introduced.

Ref⁶ introduces a low-cost 5-DOF robotic arm designed for small industries. Developed using SOLIDWORKS and analyzed via ANSYS FEA, it demonstrates structural integrity under a 1 kg payload. This solution addresses high-cost barriers, enabling automated conveyor-based object sorting with enhanced durability and directional precision for industrial applications. Ref⁷ presents a non-invasive Brain-Computer Interface (BCI) system for intelligently controlling a robotic arm to aid physically disabled individuals. It compares AI classifiers, identifying Random Forest (76% accuracy) as optimal for decoding EEG signals to predict arm movements, acknowledging person-specific bandwidth variations that cause dataset fluctuations. Further, models a fully automated 4-DOF fruit-picking robotic arm using machine vision in⁸. Fabricated from aluminum and mild steel, it employs YOLOv8 for 96% precise object detection and an inverse kinematic algorithm, achieving a 90% success rate in harvesting within 15 s to enhance agricultural productivity for smallholders.

Ref⁹ develops and evaluates a 4-DOF robotic arm with a vacuum-based end-effector for cotton harvesting. Utilizing CNNs for boll recognition and ANN-assisted optimization, it achieved 85.71% lab and 72.04% field picking efficiency, significantly lower than manual harvesting (98.04%), due to challenges with complex plant structures. Ref¹⁰ designs a self-automated, robotic-arm-based cleaning system for solar panels. Employing an IR sensor and an embedded controller, it utilizes a silicone rubber wiper, increasing PV efficiency by 13.02% and achieving a high cleaning rate (1.86 m²/s) with low water consumption (0.31 L/m²) for optimized energy gain. The authors in¹¹ introduce a Neuronally Controlled Visual Assistive Device (NCVAD) integrating computer vision and EEG processing. It uses YOLOv8n for object detection and a gradient descent-based algorithm to guide a 3-DoF robotic arm with over 85% EEG classification precision, establishing a proof-of-concept for intuitive assistive technologies for the visually impaired. Ref¹² proposes a deep learning-based approach utilizing Mask R-CNN to automatically detect and separate rotten fish by analyzing eye images with 96.5% accuracy. This system ensures export quality control by triggering a robotic arm to remove faulty products from the conveyor line, mitigating economic and health risks. Ref¹³ investigates a novel thin-film object recognition method using a piezoelectric actuator-sensor pair. Vibration frequency responses through objects are analyzed by a trained Long Short-Term Memory (LSTM) network, achieving high accuracy in transparent object classification and sheet counting tasks where traditional vision systems typically fail.

Develops a highly adhesive, quasi-homogeneous capacitive tactile sensor using a transparent, conductive ionogel in¹⁴. This sensor exhibits superior interlayer bonding strength, a broad sensing range up to 5 MPa, high linearity, and reliability over 2500 shear cycles, enhancing performance in robotic object identification and human-machine interaction interfaces. Ref¹⁵ presents a flexible, 3D-printed two-axis force triboelectric nanogenerator (TF-TENG) for self-powered sensing. Operating in contact-separation mode, its outputs 160 V and 5.5µA, functioning as an artificial skin for an emergency robot-arm stop system upon collision detection, enabling safety without spatial restrictions. Ref¹⁶ integrates a 3-DOF linear robotic arm with a deep-learning perception module (YOLOv4-tiny) for cotton harvesting. Evaluation showed a closed-loop control algorithm successfully harvested 72% of seed cotton with an 8.8s cycle time, demonstrating potential for multiple seasonal pickings to improve fiber quality and soil health. Ref¹⁷ fabricates a piezoelectric-effect-enhanced triboelectric nanogenerator (CTB-TENG) using Ecoflex doped with T-BTO and CS. Post-polarization, it achieves 532 V and 72µA output. An integrated CNN-based deep learning model enables 98.6% accurate object recognition for active tactile sensing and passive control in intelligent sorting systems. Ref¹⁸ describes an automated metering mechanism featuring a 3-DOF serial robotic arm and an LDR-LED controlled conveyor for transplanting vegetable seedlings. The system handles 20 seedlings per minute with 92.86% laboratory efficiency, offering a simple, lightweight solution to mechanize transplanting without damaging biodegradable pots. Ref¹⁹ designs an intelligent electropneumatic mechatronic conveyor system with two separated lines for transporting and positioning circular objects. Controlled by a mini-PLC and a color sensor, its novel 3D structure ensures reliable accuracy in color detection and positioning for manufacturing and packaging assembly lines.

Ref²⁰ aims to develop an automatic pick-and-place robot system for harvesting pumpkins. It employs AI (YOLO variants) for > 99% detection accuracy and an RGB-D camera for 3D localization. Field tests showed a > 90% picking success rate, though performance was hindered by fruits entangled in dense vines. Ref²¹ experimentally achieves real-time position control for a symmetric mobile robot using PID, fuzzy logic, and a YOLO-based convolutional neural network for multi-obstacle classification. The robot autonomously reaches target points from an Android application, avoiding obstacles with high detection accuracy and minimal linear speed error (2.8 mm/s). Further, Table 1 provides a general comparison of the literature presented in this section.

Table 1 Comparison of the relevant literature discussed in the introduction section.

The integration of Tiny Machine Learning (TinyML) into various applications is rapidly expanding, driven by the need for intelligent solutions in resource-constrained environments. In the following, explores recent advancements in TinyML, focusing on its application in robotics, industrial automation, and embedded systems, showcasing the versatility and potential of this technology.

Lin et al.²² developed a real-time bolt-defect detection system for climbing robots using TinyML. Employing the FOMO algorithm, the system achieves 82% average accuracy in identifying bolt conditions. This approach demonstrates suitability for resource-constrained microcontrollers, highlighting TinyML’s potential in enhancing inspection efficiency and scope in challenging environments. The system improves accuracy and efficiency over traditional methods. Yang et al.²³ proposed a TinyML-based method for in-pipe feature detection in miniature robots. A custom CNN, optimized for limited resources, achieved 97.1% accuracy in identifying key pipeline features. This demonstrates high computational efficiency and stable performance, showcasing TinyML’s effectiveness for autonomous in-pipe exploration and inspection in resource-limited settings.

Hu et al.²⁴ integrated TinyML and edge computing for real-time object recognition in industrial robotic arms. Using SparkFun Edge and Raspberry Pi Camera Module 3, the system achieved improved recognition accuracy and operational speed. This demonstrates TinyML’s potential in enhancing robotic arm intelligence, improving flexibility and automation in production processes with ultra-low energy consumption. Immonen and Hämäläinen²⁵ reviewed resource optimization challenges in TinyML for microcontrollers. They discussed quantization, pruning, and clustering methods, summarizing TinyML frameworks, libraries, and development tools. Benchmarking challenges and emerging techniques for data privacy and security were also examined, providing a comprehensive overview of TinyML’s current state and future development.

Albanese et al.²⁶ presented a sensor system with three MCU-based TinyML cameras for automatic artifact and anomaly detection in plastic components. Evaluating MobileNetV2 and SqueezeNet, the system achieved 99% classification accuracy with real-time performance. This demonstrates the feasibility of TinyML for high-accuracy product quality inspection, boosting yield and reducing costs in large-scale production. de Prado et al.²⁷ proposed a closed-loop learning flow for autonomous driving mini-vehicles using TinyCNNs. The system adapts to dynamic environments, minimizes energy consumption, and outperforms other implementations on traditional MCUs. This demonstrates the robustness of TinyML for autonomous systems, addressing challenges posed by limited on-board resources.

Navardi et al.²⁸ introduced MetaTinyML, a metareasoning framework for self-guided navigation on TinyML platforms. The framework adapts decision-making based on environmental changes, enhancing power consumption by up to 50% on an NVIDIA Jetson Nano system. This showcases the potential of metareasoning in optimizing TinyML performance. Khatoon et al.²⁹ integrated TinyML, remote sensing, and fuzzy logic through a TinyML-U-Net-FL architecture for road anomaly detection. The framework achieved high recall (92.4%), precision (78.2%), and F1-Score (84.7%), demonstrating superior performance in resource-constrained environments. This facilitates precise, energy-efficient, timely detection of road network irregularities for enhanced infrastructure management. Waseem et al.³⁰ integrated TinyML-based tack weld detection on a Renesas EK-RA6M5 platform running micro-ROS for robotic welding. The model reached an estimated F1-score of 98.7%, with minimal resource use, confirming its viability for edge analytics. This enhances autonomy and provides a foundation for advancements in robotic welding.

Bai and Yu³¹ proposed a lightweight emotion analysis solution using TinyML for portable devices. The model combines attention mechanisms and binary operations, achieving advanced performance with 70 K parameters and 0.96 MB model size. This demonstrates its superiority in terms of inference speed, enabling emotion analysis on resource-limited devices. Pleterski et al.³² implemented object classification using a CNN based on an ultralow-resolution ToF sensor on a miniature mobile robot. The system, using a VL53L5CX ToF sensor and RP2040 microcontroller, achieved 91.8% accuracy in detecting a mobile robot. It requires minimal RAM and energy consumption.

While the aforementioned works present significant advancements in various aspects of robotics, including specialized robotic arm designs^6,8,9,16,18, advanced sensing technologies^{7,13,14,15,17}, and sophisticated control or perception algorithms^{10,11,12,19,20,21,22,23,24,26,27,28,29,30,31,32}, many either focus on isolated subsystems or rely on more powerful, often off-board, computational resources. For instance, some studies utilize high-performance PCs for deep learning inference^12,20, while others focus on specific tasks like harvesting^8,9,16 or cleaning¹⁰ without emphasizing full system integration on constrained hardware. Our work distinguishes itself by unifying real-time machine learning-based object detection (YOLOv7-tiny), geometric inverse kinematics, and robust motor control (VSPHM) into a single, cohesive system operating entirely on a low-cost, resource-constrained AMB82-Mini microcontroller. This comprehensive integration on an embedded platform for a 6-DoF pick-and-place task, as summarized in Table 1, represents a significant contribution towards developing fully autonomous, cost-effective, and scalable robotic solutions, addressing the gap in achieving high-performance robotics with minimal hardware footprint and computational overhead.

Materials and methods

This section elaborates on the comprehensive methodology employed to design, develop, and evaluate the integrated 6-DoF robotic arm system with machine learning-based object detection. The system architecture, encompasses three core components: the mechanical design of the robotic arm, the electronic hardware and sensing suite, and the software algorithms for perception and control.

Fig. 1

The 3D-Printed 6-DoF Robotic Arm.

Fig. 2

The 6-DoF robotic arm components separately. (a) First printed part, (b) Second printed part, (c) Third printed part, (d) Fourth printed part, (e) Fifth printed part, (f) Sixth printed part, c) Seventh printed part, (g) Internal gears in the seventh part, (i) Connection of internal gears in the seventh part, (j) Eighth printed part, (k) The components of the eighth part, (l) Connecting the internal gears, servo motor and fingers in part eight.

Mechanical design and hardware components

The robotic manipulator was designed from the ground up to be lightweight, rigid, and capable of performing precise pick-and-place operations.

Robotic arm structure

The 6-DoF articulated robotic arm was designed using AutoCAD for computer-aided design (CAD). The non-metallic parts of the arm structure were subsequently fabricated using a Creality K1C 3D printer with PLA + filament, chosen for its good strength-to-weight ratio. The final physical implementation is shown in Fig. 1. Further, the 6-DoF robotic arm components separately is presented in Fig. 2.

Denavit-Hartenberg (D-H) parameters

The kinematic structure of the arm^33,34,35,36 was formally defined using the standard Denavit-Hartenberg (D-H) convention, which provides a systematic method for representing the spatial geometry of robotic manipulators. The D-H parameters for each joint, which are fundamental for both forward and inverse kinematic computations, are summarized in Table 2. Furthermore, to initiate the process, one must begin by constructing the kinematic diagram of the robotic arm, which requires the careful identification of all joints and links that play a vital role, as illustrated in Fig. 3.

Table 2 Denavit-Hartenberg (D-H) parameters of the designed 6-DoF robotic arm.

Fig. 3

Drawing the kinematic diagram.

Actuation system

Six servo motors were employed to actuate the six joints of the robotic arm. To achieve precise closed-loop control over each joint, the standard servos were modified. An additional feedback pin was added to access the internal potentiometer readings, providing real-time data on the actual joint angle. This modification is critical for comparing the desired angle (from inverse kinematics) with the actual angle for accurate PID-based control. A high-torque 150 kg-cm servo was used for the base joint to handle the maximum load, while 20 kg-cm servos were utilized for the subsequent joints. A modified servo is depicted in Fig. 4.

Fig. 4

Servo Motor with an additional feedback pin for real-time angle reading.

Fig. 5

The designed End-Effector with adaptive grasping capability.

End-Effector design

A custom two-fingered gripper was designed as the end-effector. It incorporates a Hall-effect-based current sensor to measure the operational current drawn by the gripper’s servo motor. This current measurement is directly proportional to the torque being applied. This functionality allows the system to implement adaptive grasping; the gripper can adjust its force based on the measured current to securely hold objects of varying weights without causing damage and slipping, as shown in Fig. 5.

Main control unit (AMB82-Mini Microcontroller)

Figures 6 and 7 show the AMB82-Mini microcontroller module and its pinout, respectively. The AMB82-Mini microcontroller, based on an ARM Cortex-M0 + core (LPC82x series), was selected as the central processing unit for this system. Key features that motivated its selection include:

• Adequate for running optimized ML models and kinematic calculations.

• Ideal for embedded applications.

• Multiple PWM outputs for servo control, UART for serial communication, and USB for programming and data transfer.

• Provides a low-cost solution for a complex robotic system. The microcontroller is responsible for image acquisition, running the object detection inference, calculating inverse kinematics, and generating PWM signals for servo control. Figure 5 shows the AMB82-Mini board.

Fig. 6

The AMB82-Mini microcontroller.

Fig. 7

The AMB82-Mini microcontroller module pinout.

Vision sensor

A full-HD CMOS image sensor (JXF37) was mounted on the end-effector to provide a first-person view for object detection and localization. This configuration allows the camera to move with the arm, enabling dynamic tracking of objects.

Software and algorithmic framework

The software pipeline is the core intelligence of the system, integrating machine learning for perception with robotic kinematics for control.

Machine learning model for object detection

The cornerstone of the system’s perception is a deep learning-based object detection model. The YOLOv7-tiny architecture was selected as the optimal compromise between high inference speed and acceptable accuracy, a critical requirement for real-time performance on the resource-constrained AMB82-Mini microcontroller³⁷.

Model architecture and optimization

The YOLOv7-tiny network employs a backbone featuring stacked convolutional and efficient layer aggregation network (ELAN) layers for feature extraction. The head utilizes path aggregation network (PANet) for multi-scale feature fusion, enabling effective detection of objects at various sizes. Key optimization strategies for deployment on the microcontroller included:

• The trained 32-bit floating-point model was converted to an 8-bit integer (INT8) model using TensorFlow Lite’s post-training quantization. This process reduces the model size by approximately 75% and significantly accelerates inference on hardware without native FPU support, with a minimal loss in accuracy.

• The model was carefully designed using only operators supported by the TensorFlow Lite Micro interpreter and the specific hardware accelerators (if any) of the ARM Cortex-M0 + core.

Dataset curation and preprocessing

A robust dataset is fundamental for model generalization. A custom dataset of over 2,500 images was meticulously collected. The images primarily featured plastic bottles, specifically mineral water bottles, which were the target objects for the pick-and-place tasks in our laboratory setup. These images were captured under highly varied conditions to ensure robustness and generalization capabilities, including:

• Different intensities, angles, and shadows to simulate diverse lighting scenarios.

• Clean backgrounds to enhance model robustness.

• Partial occlusion of objects to ensure reliable detection in non-ideal scenarios.

• Objects were captured from multiple angles and distances.

To artificially expand the dataset and improve generalization, extensive online augmentation was applied during training, including geometric transformations (random rotation (± 15°), scaling (0.8–1.2x), shear (± 10°), and translation (± 20%)), photometric transformations (adjustments to brightness (± 30%), contrast (± 20%), saturation (± 20%), and hue (± 0.1)), and noise injection (adding Gaussian noise to simulate sensor noise). All images were annotated manually using the LabelImg tool, generating bounding box coordinates (in PASCAL VOC format) and class labels for each object. Representative examples of these training images, including various conditions and annotations, are shown in Fig. 8.

Fig. 8

Examples from the custom training dataset for the YOLOv7-tiny object detection model.

Training protocol and metrics

• The model was trained on an NVIDIA GTX 3080 Ti GPU using the PyTorch framework. The training hyperparameters were set as follows: an initial learning rate of 0.01 with a cosine annealing scheduler, a batch size of 32, a momentum of 0.937, and a weight decay of 0.0005. Training was conducted for 300 epochs.

• The model optimizes a composite loss function, consisting of:

  • Objectness Loss (BCWithLogitsLoss): To determine if a cell contains an object.

  • Classification Loss (BCWithLogitsLoss): To predict the correct class of the object.

  • Bounding Box Regression Loss (CIoU Loss): To accurately predict the bounding box coordinates. The Complete-IoU loss considers overlap, center point distance, and aspect ratio, leading to superior convergence compared to standard MSE loss.

• Model performance was rigorously evaluated on a held-out test set using the following metrics:

  • The fraction of correctly identified positive predictions among all positive predictions (TP / (TP + FP)).

  • The fraction of correctly identified positive predictions among all actual positives (TP / (TP + FN)).

  • The harmonic mean of Precision and Recall (2 * (Precision * Recall) / (Precision + Recall)).

  • The primary metric for object detection, representing the area under the Precision-Recall curve averaged over all classes at an Intersection-over-Union (IoU) threshold of 0.5.

Object localization and coordinate Estimation

The coordinate of objects was calculated in real time operation from input video of AMB82-Mini. The x and y were measured form the position of object rectangles detection:

$$X\,=\,Xmin{\text{ }}+{\text{ }}\left( {\left( {Xmax{\text{ }}-{\text{ }}Xmin} \right)/2} \right)$$

(1)

$$Y\,=\,Ymin{\text{ }}+{\text{ }}\left( {\left( {Ymax{\text{ }}-{\text{ }}Ymin} \right)/2} \right)$$

(2)

Regarding Z coordinate, was measured by using mapping function and depended on machine learning processing and what the data trained on.

$$L\,=\,Xmax{\text{ }}-{\text{ }}Xmin$$

(3)

$$z{\text{ }}={\text{ }}\left( {L{\text{ }}-{\text{ }}Lmax} \right){\text{ }}*{\text{ }}\left( {Zmax{\text{ }}-{\text{ }}Z{\text{ }}min} \right){\text{ }}/{\text{ }}\left( {Lmin{\text{ }} - {\text{ }}Lmax} \right)\,+\,Zmin$$

(4)

The mathematical expression represented by Eq. (4) delineated the specific mapping function that corresponds between different ranges of values. In this context, L represents the width of the rectangle utilized for object detection. Lmax denotes the maximum width recorded, whereas Lmin indicates the minimum width. Additionally, Zmax refers to the highest value documented in practical terms, and Zmin corresponds to the lowest value recorded in practical terms.

The z coordinate is determined based on the width of the L parameter, which represents the width of the rectangular object, calculated as the difference between Xmax and Xmin. This value of L is then compared with the training data utilizing machine learning techniques. Table 3 illustrates the mapping range ratio for selecting the z-coordinate of the target object.

Table 3 The z-coordinate range ratio.

Further, Fig. 9. Schematic diagram illustrating the geometric principles for estimating object coordinates from 2D bounding box detections. The diagram visually depicts the camera’s perspective, the target object (mineral water bottle) in 3D space, its projection onto the image plane with the detected bounding box, and the key parameters and variables (focal length f, real-world object width $\:{W}_{\left\{real\right\}}$, corresponding pixel width $\:{w}_{\left\{pixel\right\}}$, and the estimated depth Z).

Fig. 9

Geometric principles for 3D object coordinate estimation from 2D bounding boxes.

In this control algorithm, the equation is developed relating to the robot arm in the form of inverse kinematics and then send to servo motor to achieve the rotation angle. The arm manipulator control system is designed to implement a 6-DoF robot arm, and the goal of this design is to slave the end-point node of the robot to follow the specified trajectory. Therefore, an instantaneous inverse kinematic technique must be used to carry out trajectory tracking from desired x, y z and coordinates. Basically, robot control involves the design of a set of control laws used to fire the actuators on the robot joints to ensure that the end-effector follows the given desired trajectory.

Inverse kinematics (IK) solver

A geometric inverse kinematics approach was implemented specifically for the kinematic structure defined by the D-H parameters in Table 2. The IK algorithm takes the desired (X, Y, Z) Cartesian coordinates of the end-effector (calculated from object detection) as input and computes the six required joint angles ($\:{\theta\:}_{1}$ to $\:{\theta\:}_{6}$). These angles are then translated into target PWM signals for each servo motor.

Control logic and system integration

The integration of these components into a deterministic control loop is critical for system stability. The control task on the AMB82-Mini was structured as a fixed-priority preemptive loop. The highest priority was given to servo feedback reading and PID control to prevent jitter and ensure smooth motion. The medium priority was assigned to the main control logic and IK calculations. The lowest priority was given to the image acquisition and object detection inference, which, due to its variable execution time, is run asynchronously. Further, the system operates via a finite state machine (FSM):

• 1. Waits for a start command.

• 2. Captures an image and runs inference. If an object is detected with confidence > 0.8, proceeds; otherwise, returns to IDLE.

• 3. Computes the 3D coordinates and solves for the target joint angles.

• 4. Commands the servos to the target angles using a trapezoidal velocity profile for smooth motion. The PID controller for each joint uses the feedback from the modified servos to minimize the error between the desired and actual angle.

• 5. Activates the gripper. The Hall-effect current sensor monitors the draw. Gripping continues until a current threshold (indicating sufficient force has been applied) is reached or a timeout occurs.

• 6. Moves the arm to a predefined drop-off location and releases the object.

Further, coordinates and commands between the detection, IK, and control modules are passed via thread-safe queues to manage the asynchronous nature of the vision pipeline versus the synchronous control requirements.

Results and discussion

This study successfully designed, implemented, and validated a fully integrated 6-DoF robotic arm system featuring onboard machine learning-based object detection and advanced inverse kinematics (IK) control, all executed on the embedded AMB82-Mini microcontroller. Further, Fig. 10 shows the hardware components circuit diagram. The following sections present a detailed analysis and discussion of the experimental and simulation results obtained for each core component of the system.

Performance of embedded machine learning object detection

A pivotal achievement of this work was the development and deployment of a real-time object detection and classification model on the resource-constrained AMB82-Mini platform. The choice of YOLOv7-tiny was strategic; its architecture provides an optimal balance between speed and accuracy, making it uniquely suited for embedded systems where computational power and memory are limited. The model was trained on a custom dataset of over 2,500 manually captured images (as detailed in Sect. 3.2.1.2). The training process was highly effective, with both training and validation loss demonstrating a consistent and concurrent decline. This parallel decrease, as shown in the loss curves, is a critical indicator of a well-generalized model. It conclusively demonstrates that the model was learning relevant features from the data without memorizing the training set (overfitting) or failing to learn the underlying patterns (underfitting). The final convergence of both curves to a low value confirms the model’s readiness for deployment.

For the quantized YOLOv7-tiny model, deployed on the AMB82-Mini microcontroller, we achieved a Mean Average Precision (mAP@0.5) of 88.2% on the held-out validation set. This metric, which considers both precision and recall at an Intersection-over-Union (IoU) threshold of 0.5, confirms the model’s high accuracy in detecting and localizing objects under various conditions. This performance is achieved through a lean processing, where the YOLOv7-tiny model’s forward pass on 640 × 480 pixel images contributes approximately 120–130 ms to the total processing time, including pre- and post-processing (NMS and 3D coordinate estimation). This end-to-end processing time, translating to an average of 7–8 FPS, is well-suited for the dynamic requirements of robotic pick-and-place operations on a resource-constrained embedded platform. It ensures the robot can react dynamically to detected objects with sufficient temporal resolution for effective manipulation tasks. This performance is critical for enabling the system’s real-time object localization and subsequent pick-and-place operations on a resource-constrained embedded platform. This speed, combined with the low latency of our control loop, ensures the robot can react dynamically to detected objects.

Quantization, the process of converting the model’s weights from floating-point to lower-precision integers, was essential for achieving operational speed on the microcontroller. While this conversion can sometimes lead to a minor loss in accuracy, the results confirm its effectiveness. The quantized model, running on the microcontroller’s dedicated inference engine, reliably captures images via its integrated camera and outputs the precise Cartesian coordinates (x, y, z) of the centroid of detected objects, as illustrated in the output screenshot in Fig. 11. This coordinate data, generated entirely on-board, serves as the primary and robust input for the subsequent robotic pick-and-place tasks. This capability to perform accurate, real-time inference without relying on external computational resources like a desktop PC or cloud server represents a significant advancement in creating autonomous, low-cost, and portable robotic systems.

Fig. 10

Hardware components circuit diagram.

Accuracy and efficacy of the implemented inverse kinematics solution

The inverse kinematics algorithm, implemented in the Arduino IDE, forms the core of the robot’s motion control. The algorithm’s robustness is derived from its rigorous mathematical foundation based on the DH convention, which provides a systematic method to describe the geometric relationships between consecutive joints. The transformation matrix for the sixth joint (T₆), as expressed in Eq. (5), is fundamental to this description. The calculation of the first joint angle, J1, is a critical step that demonstrates the algorithm’s sophistication. Rather than a simple arctangent calculation, it employs a quadrant-aware analysis of the matrix T₅ (Eqs. (6)-(9)). This conditional logic is necessary to resolve the inherent ambiguity in the arctangent function, ensuring that the angle is calculated correctly across all four quadrants of the Cartesian plane and preventing catastrophic positioning errors. Figure 12 shows the top/side view of joints J1, J2, and J3.

Fig. 11

The (x, y, z) coordinate output of a detected object from the microcontroller’s camera inference engine.

The subsequent calculations for J2 and J3 (Eqs. (10)-(21)) involve a detailed trigonometric analysis of the projected arm geometry (see Fig. 12). The solution effectively breaks down the complex 3D problem into 2D sub-problems in the vertical and horizontal planes. The use of the Law of Cosines to find angles θC and θD is a classic and reliable approach for solving the triangles formed by the robot’s links (a₂, L₂, L₃). The final three joint angles (J4, J5, J6), which control the end-effector’s orientation, were solved by obtaining the inverse of the combined transformation matrix from the first three joints. The use of the atan2 function in Eqs. (22)-(24) is again crucial here. Unlike a standard atan function, atan2 uses the signs of both input parameters to determine the correct quadrant of the resulting angle, ensuring the wrist joints are oriented correctly without ambiguity. The entire IK solution proved to be computationally efficient enough to run on a microcontroller, enabling the robot to reach desired positions within its workspace reliably and with high accuracy, as validated by the successful pick-and-place operations.

Fig. 12

Top/side view representation of joints J1, J2, and J3 for kinematic analysis.

Inverse kinematics accuracy and robustness analysis

To quantitatively assess the precision and reliability of our IK algorithm, as detailed in Sect. 4.2, we conducted a simulation-based error analysis. This approach models the real-world performance by comparing theoretical calculations with positions derived from simulated noisy joint actuations. We defined the robot’s link lengths ($\:{L}_{1}=100$ mm, $\:{L}_{2}=100$ mm, $\:{L}_{3}=50$ mm) consistent with our physical arm. A total of 1000 random desired end-effector positions ($\:{P}_{x},{P}_{y}$) and orientations ($\:\varphi\:$) were generated within the robot’s approximate reachable workspace. For each desired pose, the theoretical joint angles ($\:{\theta\:}_{1},{\theta\:}_{2},{\theta\:}_{3}$) were calculated using our geometric IK solution. To simulate real-world execution inaccuracies, a small amount of Gaussian noise with a standard deviation of 0.5 degrees (approx. 0.0087 radians) was added to each of these theoretical joint angles. Subsequently, Forward Kinematics (FK) was applied to these noisy joint angles to determine the “actual” end-effector position. The positional error was then calculated as the Euclidean distance between the desired and the simulated actual end-effector positions.

The results of this simulation provide a robust estimation of the IK system’s performance under realistic conditions. For 1000 trials, the analysis yielded a Mean Positional Error of approximately 0.8251 mm, a Maximum Positional Error of 3.1205 mm, and a Root Mean Square Error (RMSE) of 1.0267 mm. These metrics demonstrate the inherent precision of our IK algorithm and its robustness against typical joint actuation inaccuracies. The RMSE, in particular, provides a comprehensive measure of the overall positional accuracy, indicating that the end-effector can be positioned with high reliability within a small error margin, which is crucial for precise pick-and-place operations. Figure 13 visually represents the desired versus simulated actual positions, illustrating the distribution of these positional errors across the workspace.

Fig. 13

Simulated IK positional accuracy and error analysis. (a) Overall view of the robot’s workspace, illustrating the distribution of desired end-effector positions (blue circles) and the corresponding actual positions (red crosses) resulting from simulated noisy joint actuations. (b) Zoomed-in subset of the workspace, specifically highlighting the positional error vectors (green arrows) between desired and actual positions.

Functionality of the smartphone control application

The MIT App Inventor 2 platform was chosen for its accessibility and rapid development capabilities, which are ideal for creating functional prototypes and educational tools. The developed application, with its dual-mode interface shown in Fig. 14, provides two distinct paradigms for interaction, catering to different user needs and demonstrating the system’s flexibility. The Inverse Kinematics (IK) Mode is the higher-level mode, allowing users to command the robot in task-space by specifying the desired end-effector pose. This abstracts away the complexity of individual joint control and is the mode used for autonomous operation based on camera input. The Forward Kinematics (FK) Mode, on the other hand, provides low-level joint control. This mode is invaluable for debugging, manual calibration, and educational purposes, as it allows users to directly see the effect of each joint on the overall pose of the robot.

Fig. 14

Smartphone application interface: (a) Inverse Kinematics (IK) control mode; (b) Forward Kinematics (FK) control mode.

The reliable Bluetooth Low Energy (BLE) communication protocol ensures a stable wireless connection. The application’s ability to display real-time feedback messages from the robotic arm is a key feature that enhances usability, providing immediate confirmation of command execution and alerting the user to any system status changes, thus creating a closed-loop interaction between the user and the machine.

Voltage source parallel hysteresis modulation (VSPHM)

The Voltage Source Parallel Hysteresis Modulation (VSPHM) is a critical component of our low-level motor control strategy, primarily implemented within the motor driver stage for each joint. This modulation technique was selected for its inherent robustness, fast dynamic response, and its effectiveness in mitigating chattering, which are essential characteristics for stable and precise motor control in resource-constrained embedded systems.

The “parallel” aspect of VSPHM signifies its integration within a hierarchical control architecture. While higher-level controllers determine the desired current or voltage commands for the motors based on position and velocity feedback, the VSPHM operates at the power electronics level. It translates these continuous commands into discrete switching signals for the voltage source, ensuring that the actual motor current or voltage closely tracks the commanded value with minimal ripple and stable operation.

The core principle of hysteresis modulation involves a comparator with a predefined hysteresis band. For a given error signal, $\:e\left(t\right)$, which represents the difference between the desired and actual motor parameter (e.g., current), and a hysteresis band of total width $\:2H$ (with upper threshold $\:{H}_{upper}$ and lower threshold $\:{H}_{lower}$), the switching logic for the output voltage $\:{V}_{out}$ is as follows:

Let $\:e\left(t\right)$ be the error signal (e.g., $\:{I}_{desired}-{I}_{actual}$). Let $\:H$ be the half-width of the hysteresis band, such that $\:{H}_{upper}=H$ and $\:{H}_{lower}=-H$.

The switching logic is defined by: * If $\:e\left(t\right)>{H}_{upper}$ and the previous state was $\:{V}_{out}={V}_{min}$ (or off), then switch to $\:{V}_{out}={V}_{max}$. If $\:e\left(t\right)<{H}_{lower}$ and the previous state was $\:{V}_{out}={V}_{max}$, then switch to $\:{V}_{out}={V}_{min}$ (or off). * If $\:{H}_{lower}\le\:e\left(t\right)\le\:{H}_{upper}$, the current state of $\:{V}_{out}$ is maintained.

This mechanism inherently limits the switching frequency, preventing excessive high-frequency oscillations (chattering) that can occur with simple bang-bang control, while simultaneously ensuring a rapid response to larger error deviations. The selection of the hysteresis band width, $\:2H$, is crucial; a narrower band results in higher switching frequency and lower ripple but increased switching losses, while a wider band reduces switching frequency and losses at the cost of higher ripple. For our application, $\:H$ was empirically tuned to balance these factors, prioritizing stability and chattering reduction.

Comprehensive evaluation of servo motion smoothing algorithms

Figure 15, providing a comprehensive overview of various easing functions used for generating smooth, non-linear transitions. Each subplot illustrates the output value (Y-axis) as a percentage of the total transition, corresponding to the input progress (X-axis) from 0% to 100%. These functions are categorized into three primary types: (a) Ease-InOut, (b) Ease-Out, and (c) Ease-In, each designed to control the acceleration and deceleration profile of a motion or animation. The diverse color palette and clear axis labels enhance readability, allowing for direct comparison of their characteristic response curves.

Subplot (a) displays Ease-InOut functions, which provide a smooth start, accelerate through the middle, and then decelerate towards the end, creating a balanced and natural transition. Subplot (b) presents Ease-Out functions, characterized by a rapid initial acceleration that gradually slows down towards the completion of the transition, making them suitable for actions that require an immediate strong response followed by a gentle finish. Conversely, subplot (c) illustrates Ease-In functions, which begin slowly and progressively accelerate towards the end, ideal for actions that build momentum. Functions like Elastic and Bounce exhibit oscillatory behaviors, while Back (Overshoot) demonstrates a slight overshoot before settling, offering unique motion profiles beyond simple acceleration/deceleration. Finally, Fig. 16 presents the response (x, y and z) coordinates from microcontroller camera.

Fig. 15

Comparative response of the smoothing equations. (a) Ease-InOut, (b) Ease-Out, (c) Ease-In.

Overall system integration and performance validation

The ultimate test of the system’s efficacy was its performance in integrated pick-and-place experiments under varying payload conditions, as summarized in Table 4. The results demonstrate a robust system capable of handling a significant range of weights. The perfect success rate for payloads up to 300 g validates the synergy between the mechanical design, the accuracy of the IK solution, and the stability provided by the motion smoothing.

Fig. 16

The Response (x, y and z) coordinates from microcontroller camera.

Table 4 System performance evaluation under different payload conditions.

To provide a more robust statistical analysis of the system’s performance, each pick-and-place task for every payload condition was repeated 20 times (N = 20). The ‘Time (s)’ and ‘Operation Power (W)’ values reported in Table 4 represent the average (Avg.) and standard deviation (SD) over these 20 trials. The ‘95% Confidence Interval’ for the success rate was calculated using the Wilson Score method, which is appropriate for proportion data, especially with a limited number of trials. The experiments were conducted under consistent indoor lighting conditions to ensure controlled testing for payload variations. However, it is crucial to note that the underlying object detection model was trained on a custom dataset specifically curated with highly varied lighting conditions (different intensities, angles, and shadows) and diverse backgrounds (cluttered and clean), as detailed in Sect. 3.2.1.2. Throughout the testing, standard laboratory temperature (26 °C) and humidity (40% RH) were maintained.

The slight increase in operation time and power consumption with heavier payloads is an expected and physically justified result. The servo motors must exert more torque to lift heavier objects. The implemented control logic correctly responds by moving more slowly and deliberately to maintain stability and avoid slipping or dropping the object, which explains the increased time. The higher power draw is a direct correlate of the increased current required to generate this higher torque. The minor drop-in success rate at 400 g and 500 g is likely due to factors like slight flex in the robot’s links or nearing the torque limits of the servos, which is a known constraint of low-cost hardware. This trend actually reinforces the validity of the data, showing the system behaving predictably under stress.

Quantitative comparison of VSPHM with baseline control methods

To better evaluate the improvements offered by the VSPHM scheme, we compared its performance against two baseline methods: a standard PID controller (tuned with gains Kp = 1.5, Ki = 0.2, Kd = 0.1, as per initial system calibration) and a basic hysteresis modulation (with a fixed hysteresis band of ± 5 V, without parallel voltage source adaptation). These baselines represent conventional approaches for handling nonlinear actuator behaviors in resource-constrained systems like the AMB82-Mini, but they often suffer from chattering or instability under varying payloads.

The comparisons were executed in both simulation and physical experiments (20 trials per method, with a 200 g plastic bottle payload and 100 mm distance under controlled indoor conditions, as in Table 4). Key metrics include:

• RMSE (mm): Measures average positioning error across joint trajectories.

• Overshoot (%): Peak deviation from the target position.

• Convergence Time (s): Time to reach steady state (within 1% of target).

• Chattering Amplitude (V): Peak-to-peak fluctuations in servo voltage signals, indicating control instability.

Further, quantitative comparison of VSPHM with baseline control methods (200 g payload, N = 20 trials) are summarized in Table 5. VSPHM demonstrates superior performance, reducing RMSE by 45–60%, eliminating overshoot entirely, shortening convergence time by 30–50%, and minimizing chattering by 70–80% compared to baselines. These gains stem from VSPHM’s adaptive tuning and parallel voltage sourcing, which mitigate electromechanical nonlinearities (hysteresis in servos) without requiring additional hardware resources. In simulations, VSPHM’s advantages are more pronounced due to idealized conditions, while experimental results reflect real-world noise (sensor feedback inaccuracies). Notably, the standard PID exhibited persistent chattering (up to 8 V fluctuations), leading to mechanical vibrations and reduced success rates (85% vs. 100% for VSPHM at 200 g). The basic hysteresis reduced chattering somewhat but introduced overshoot and slower convergence due to lack of adaptive voltage integration.

These comparisons confirm VSPHM’s efficacy in our embedded setup, however, they are limited to the specific hardware constraints of the AMB82-Mini (limited computational throughput preventing real-time optimization of more complex baselines). A broader comparative study with advanced methods (model predictive control) could be explored in future work, potentially on higher-power platforms.

Table 5 Quantitative comparison of VSPHM with baseline control methods (200 g payload, N = 20 trials).

Limitations of the experimental validation

While the experimental results demonstrate the system’s efficacy in achieving high success rates (up to 100% for payloads ≤ 300 g) and real-time performance (7–8 FPS inference on the AMB82-Mini), the validation is subject to certain limitations that should be acknowledged for contextual interpretation. Specifically:

Single Object Type: Testing was restricted to plastic bottles as the target objects. This choice was made to focus on a consistent, lightweight, and graspable item that aligns with common industrial pick-and-place scenarios (packaging or sorting in SMEs). Plastic bottles provide a standardized shape and material for evaluating the YOLOv7-tiny model’s detection accuracy and the gripper’s mechanical reliability, minimizing variables such as deformability or surface texture that could introduce confounding factors in early-stage testing. However, this limits generalizability to diverse object types (e.g., irregular shapes, fragile items, or heavier materials).

Fixed Pick-and-Place Distance: All experiments utilized a fixed distance of 100 mm between pick and place positions. This setup was selected to isolate the effects of payload variations (100–500 g) on system performance metrics like time, power consumption, and accuracy (as detailed in Table 4), while ensuring repeatable trajectories within the robot’s workspace constraints. Varying distances could introduce additional kinematic complexities (longer paths increasing jerk or error accumulation), which were deferred to prioritize core integration testing.

Controlled Indoor Conditions: Experiments were conducted under consistent indoor lighting (standard laboratory conditions with varied backgrounds as per the trained dataset), temperature (26 °C), and humidity (40% RH). This controlled environment was essential to mitigate external noise and focus on the embedded system’s intrinsic capabilities, particularly given the resource constraints of the AMB82-Mini microcontroller (limited processing power for handling real-time variations in lighting or shadows). The model was trained on a diverse dataset incorporating varied lighting intensities, angles, and backgrounds (as described in Sect. 3.2.1.2), but real-world testing was limited to indoor settings to avoid hardware overload or safety issues during initial validation.

These limitations stem primarily from the proof-of-concept nature of this study, which prioritizes demonstrating the feasibility of unifying advanced machine learning-based perception (quantized YOLOv7-tiny), inverse kinematics, and control on a single, low-power microcontroller without external resources. The AMB82-Mini (ARM Cortex-M0 + core) imposes strict constraints on memory (model quantization to INT8 reduced size by ~ 75% but limits handling of highly variable inputs) and computational throughput, making broader testing (multi-object or outdoor scenarios) challenging at this stage. This setup allows for a focused evaluation of the system’s core innovations real-time embedded integration and stability while providing a foundation for scalability.

As future work, we propose expanding validation to include diverse object types (metal cans or fruits), variable distances (up to the arm’s full workspace of ~ 400 mm), and uncontrolled environments (outdoor lighting or cluttered industrial settings) using enhanced hardware or model optimizations. These extensions could further validate the system’s robustness and address the current scope limitations.

Discussion and comparative analysis

The results confirm that the primary objective of creating a low-cost, efficient, and fully integrated robotic system has been achieved. The key innovation lies in the consolidation of all computational tasks real-time object detection, complex IK calculations, and sophisticated servo control onto a single, low-power microcontroller (AMB82-Mini). This stands in contrast to most existing academic and hobbyist projects, which typically offload intensive tasks like machine learning and kinematics to an external computer, communicating via serial commands to a simpler microcontroller for basic motor control. Our system eliminates this dependency, resulting in a more compact, power-efficient, and truly standalone solution.

The modular nature of both the hardware and software design makes this system an excellent platform for educational purposes. It allows students and researchers to experiment with every aspect of a modern robotic system from embedded AI and computer vision to classical mechanics and control theory in an integrated and hands-on manner. Finally, this work demonstrates a practical and effective implementation of an autonomous 6-DoF robotic arm. The system performs reliably across its intended workload, showcasing the potent combination of modern embedded AI capabilities with classical robotic control principles.

Conclusion

This study successfully designed, implemented, and validated a fully integrated 6-DoF robotic arm system capable of real-time object detection and precise manipulation, all executed on the embedded AMB82-Mini microcontroller. The key achievement lies in the seamless integration of a machine learning-based vision pipeline, an accurate inverse kinematics solver, and robust servo control within a single low-cost, low-power embedded system, eliminating the need for external computational resources. The system demonstrated high performance across several metrics. The quantized YOLOv7-tiny model achieved reliable real-time object detection and accurate 3D coordinate estimation under varied conditions. The geometric inverse kinematics solver, based on D-H parameters, provided precise joint angle calculations, enabling accurate end-effector positioning. The implementation of advanced motion smoothing algorithms (In/Out-Cubic) significantly reduced jerk and mechanical vibrations, resulting in smooth and stable motion profiles. Experimental validation confirmed the system’s robustness, with a 100% success rate in pick-and-place operations for payloads up to 300 g and high success rates even at heavier loads.

This work effectively bridges the gap between theoretical research and practical implementation, offering a cost-effective, standalone solution for automated manipulation tasks. The integration of perception, planning, and control on a single microcontroller platform represents a significant advancement for embedded robotics, with direct applicability in industrial automation, educational settings, and SMEs. Future work will focus on expanding object detection capabilities, incorporating dynamic obstacle avoidance, and further optimizing computational efficiency for enhanced real-time performance.