Dynamic ball manipulation in rugged environments requires legged robots to coordinate terrain traversal and ball control under sparse learning rewards. This work proposes a hierarchical reinforcement learning framework in which a high-level policy switches among pre-trained low-level skills for ball dribbling and rough-terrain navigation. It also introduces Dynamic Skill-Focused Policy Optimization to improve learning efficiency for mixed discrete-continuous action spaces. Simulation and real-world experiments on a quadruped robot validate dynamic ball manipulation across challenging terrains.
Dynamic Formation Planning and Control for Robot Soccer Game with Multi-Agent Reinforcement Learning and Behavioral Model
The robot soccer game has been considered as an illustrative scenario to test the performance of research outcomes on multi-agent systems (MAS). While various algorithm has been developed for a robot soccer game and implemented in the RoboCup competition, relatively little collaboration can be found in existing results, e.g., the formation for passing and dribbling during the offense or the collaborative obstruction during defense, which are very common in the teamwork of human soccer game.
References
2026
Dynamic Legged Ball Manipulation on Rugged Terrains with Hierarchical Reinforcement Learning
Dongjie Zhu, Zhuo Yang, Xuesong Li, Wenjun Xu, Qi Liu, and Xiang Li
Achieving reliable object manipulation while traversing complex terrains is the missing link between agile quadruped locomotion and practical autonomy. Specifically, using traditional end-to-end reinforcement learning (RL) for dynamic ball manipulation in rugged environments presents two key challenges. The first is coordinating distinct motion modalities to integrate terrain traversal and ball control seamlessly. The second is overcoming sparse rewards in end-to-end RL, which impedes efficient policy convergence. To address these challenges, we propose a hierarchical RL framework. A high-level policy, informed by proprioceptive data and ball position, adaptively switches between pre-trained low-level skills such as ball dribbling and rough terrain navigation. We further propose Dynamic Skill-Focused Policy Optimization to suppress gradients from inactive skills and enhance critical skill learning. Both simulation and real-world experiments validate that our method outperforms baseline approaches in dynamic ball manipulation across rugged terrains, highlighting its effectiveness in challenging environments.
@article{zhu2026dynamic,title={Dynamic Legged Ball Manipulation on Rugged Terrains with Hierarchical Reinforcement Learning},author={Zhu, Dongjie and Yang, Zhuo and Li, Xuesong and Xu, Wenjun and Liu, Qi and Li, Xiang},year={2026},month=jun,journal={IEEE Robotics and Automation Letters},volume={11},number={6},pages={6815--6822},doi={10.1109/LRA.2026.3682979},}
