This work was developed as part of Microbotics Research Group, a research project exploring Magnetically actuated soft robots for site specific mechanical thrombectomy in coronary vasculature
The broader goal of the project was to study how small remotely actuated soft robots could eventually move through vascular structures & reach a thrombotic blockage and support precise localized intervention
The research program was divided into multiple tracks. I worked as an Associate Researcher in Track E: Navigation Control, Reinforcement Learning, and Swarm Behavior
My main responsibility was to build the foundational simulation environment for future reinforcement learning experiments
Before an RL agent can learn navigation it needs a controlled environment where it can -
- Observe the robot state
- Apply actions
- Interact with physics
- Receive feedback
I built this environment using PyBullet for 3D physics simulation and Gymnasium for the reinforcement learning interface
Since the final vascular CAD models and detailed magnetic physics from other tracks were not yet available, I followed the planned workaround and created a simplified setup using a straight cylindrical vessel and a spherical dummy microbot
The code was designed in a modular way so these placeholder components can later be replaced with more realistic vessel geometry and physics models
The GUI displays the straight vessel, the simulated microbot, and a navigation target. While running, the environment continuously reports the robot position, target distance, and reward values in the terminal
The current demo uses randomly sampled actions
The final result is a working and modular RL ready microbot simulation environment that connects PyBullet physics with the standard Gymnasium interface
My contribution focused on establishing the simulation foundation required for future microbot navigation research, RL policy training, and later integration with more realistic vascular geometry and magnetic actuation models