Title

Autonomous Control of Magnetic Microrobots and Nanoparticles Using Reinforcement Learning-Driven Electromagnetic Actuation Systems for Biomedical Applications

Alternative Title

생체의학 응용을 위한 강화 학습 기반 전자기 구동 시스템을 사용한 자성 마이크로로봇 및 나노입자의 제어

DGIST Authors

Sarmad Ahmad Abbasi ; Hongsoo Choi ; Sang Hyun Park

Advisor

최홍수

Co-Advisor(s)

Sang Hyun Park

Issued Date

2025

Awarded Date

2025-08-01

Type

Thesis

Description

Reinforcement learning, Microrobot, Nanoparticles, Magnetic Actuation system

Table Of Contents

1. Introduction 1
1.1 Background 1
1.2 Microrobots (MRs) and their actuation 2
1.3 Magnetic Actuation Systems 8
1.4 Magnetic Nanoparticles (MNPs) and their control 10
1.5 Machine Learning (ML) 13
1.5.1 Reinforcement Learning (RL) 15
1.6 Machine Learning and Microrobots 17
1.7 Machine Learning and Nanoparticles 20
2. Scope and objectives 23
2.1 Autonomous Control of Magnetic Microrobots 23
2.2 Autonomous Control of Magnetic Nanoparticles 26
3. Reinforcement Learning Based Autonomous Control of Magnetic Microrobot Using Electromagnetic Actuation System 30
3.1 Introduction 31
3.1.1 Objectives 32
3.2 Results and Discussion 33
3.2.1 Reinforcement Learning (RL) 33
3.2.2 Gradual Training Process 35
3.2.3 Simulation Environment 36
3.2.4 Training the RL agent 39
3.2.5 Comparison of RL and PID 47
3.2.6 Navigation in the middle cerebral artery phantom 51
3.2.7 Fully autonomous control via path planning 52
3.2.8 Training in fluid flow environment 56
3.2.9 Adaptability with different microrobots 59
3.3 Methods 61
3.3.1 Electromagnetic actuation system 61
3.3.2 Microrobot and Materials. 61
3.3.3 Dynamic flow environment 62
3.3.4 Simulation environment: Development and calculations 62
3.3.5 Tracking the microrobot 64
3.3.6 Training method and procedure 65
3.3.7 PID controller 69
3.3.8 Path planning for physical obstacles 69
3.4 Data and Code Availability 72
3.5 Conclusion 73
4 Sim-to-Real Reinforcement Learning Based Autonomous Magnetic Nanoparticle Swarm Locomotion 76
4.1 Introduction 77
4.1.1 Objectives 78
4.2 Results and Discussion 79
4.2.1 MNP Swarm formation and motion 79
4.2.2 Simulation environment 80
4.2.3 Reinforcement Learning (RL) 82
4.2.4 Modular Reinforcement Learning Framework 84
4.2.5 Path planner 85
4.2.6 RL agent 1: Target tracking 86
4.2.7 RL agent 2: Shape selection 89
4.2.8 Navigation in simulation 92
4.2.9 Navigation in physical system 94
4.3 Methods 98
4.3.1 Electromagnetic actuation system 98
4.3.2 Magnetic Nanoparticles (MNPs) 98
4.3.4 Simulation environment: Development and calculations 99
4.3.5 Tracking the MNP swarm 100
4.3.6 Training method and procedure 100
4.4 Conclusion 104
5 Conclusion and Future Work 106
5.1 Conclusion 106
5.2 Future work 108
ACKNOWLEDGMENT 114
PUBLICATION LIST 115
REFERENCE 117
요 약 문 137

URI

https://scholar.dgist.ac.kr/handle/20.500.11750/59774
http://dgist.dcollection.net/common/orgView/200000889898

DOI

10.22677/THESIS.200000889898

Degree

Doctor

Department

Department of Robotics and Mechatronics Engineering

Publisher

DGIST

Show Full Item Record