Autonomous 3D positional control of a magnetic microrobot using reinforcement learning

Abstract

Magnetic microrobots have shown promise in the field of biomedical engineering, facilitating precise drug delivery, non-invasive diagnosis and cell-based therapy. Current techniques for controlling the motion of such microrobots rely on the assumption of homogenous magnetic fields and are significantly influenced by a microrobot’s properties and surrounding environment. These strategies lack a sense of generality and adaptability when changing the environment or microrobot and exhibit a moderate delay due to independent control of the electromagnetic actuation system and microrobot’s position. To address these issues, we propose a machine learning-based positional control of magnetic microrobots via gradient fields generated by electromagnetic coils. We use reinforcement learning and a gradual training approach to control the three-dimensional position of a microrobot within a defined working area by directly managing the coil currents. We develop a simulation environment for initial exploration to reduce the overall training time. After simulation training, the learning process is transferred to a physical electromagnetic actuation system that reflects real-world intricacies. We compare our method to conventional proportional-integral-derivative control; our system is more accurate and efficient. The proposed method was combined with path planning algorithms to allow fully autonomous control. The presented approach is an alternative to complex mathematical models, which are sensitive to variations in microrobot design, the environment and the nonlinearity of magnetic systems. © 2024, The Author(s), under exclusive licence to Springer Nature Limited.