All projectsManipulation, reinforcement learning
Dual-arm RL platform
A bimanual platform for reinforcement learning on contact-rich tasks
Two AgileX PiPER 6-DOF arms with two-finger grippers, set up for bimanual work. Three RGB-D cameras (overhead, side and wrist level) cover the blind spots during grasps and handovers.
- Type
- Simulation
- Period
- Aug – Oct 2025
- Role
- Simulation, RL pipeline and perception
- Stack
- AgileX PiPER, Genesis, Gazebo, Python, PyTorch, MoveIt, RGB-D simulation
Challenge
RL on two arms stalls easily. Rewards are sparse, the arms collide with each other, and the policy outputs commands the arms cannot execute.
Approach
The reward mixes a sparse completion term with dense shaping: distance to goal, grasp stability, collision penalties and a coordination bonus. Observations are joint states, end-effector poses and RGB-D features. Actions are Cartesian with safety clamps, and a trajectory planner takes over with a collision-free path when the policy asks for something infeasible.
Result
A simulation platform, later reused for vision-language-action experiments and a stack-and-sort task that combines vision with the RL policy.
What I built
- Dual PiPER arms with a three-camera RGB-D perception rig
- Custom dense and sparse reward design
- Curriculum-style task progression
- Planning fallback for safe policy execution
- Stack-and-sort task combining vision and RL