ECO：基于强化学习的仿人机器人行走能量约束优化

ECO: Energy-Constrained Optimization with Reinforcement Learning for Humanoid Walking

February 6, 2026

作者: Weidong Huang, Jingwen Zhang, Jiongye Li, Shibowen Zhang, Jiayang Wu, Jiayi Wang, Hangxin Liu, Yaodong Yang, Yao Su

cs.AI

摘要

实现稳定且高能效的运动是人形机器人在现实应用中持续运行的关键。现有模型预测控制（MPC）和强化学习（RL）方法通常将能耗相关指标嵌入多目标优化框架，这需要大量超参数调优且易导致策略次优化。为解决这些问题，我们提出能量约束优化（ECO）框架，该约束强化学习方法将能耗指标从奖励函数中分离，重构为显式不等式约束。该方法为能耗成本提供了清晰可解释的物理表征，通过更高效直观的超参数调优提升能效。ECO通过拉格朗日法分别对能耗和参考运动施加专用约束，实现人形机器人稳定、对称、高能效的行走。我们在儿童尺寸人形机器人BRUCE上进行了仿真到仿真、仿真到实物的对比实验，结果表明相较于MPC、标准奖励塑形RL及四种先进约束RL方法，ECO在保持稳健步态性能的同时显著降低了能耗。这些成果标志着人形机器人能效运动控制取得重大进展。所有实验演示详见项目网站：https://sites.google.com/view/eco-humanoid。

English

Achieving stable and energy-efficient locomotion is essential for humanoid robots to operate continuously in real-world applications. Existing MPC and RL approaches often rely on energy-related metrics embedded within a multi-objective optimization framework, which require extensive hyperparameter tuning and often result in suboptimal policies. To address these challenges, we propose ECO (Energy-Constrained Optimization), a constrained RL framework that separates energy-related metrics from rewards, reformulating them as explicit inequality constraints. This method provides a clear and interpretable physical representation of energy costs, enabling more efficient and intuitive hyperparameter tuning for improved energy efficiency. ECO introduces dedicated constraints for energy consumption and reference motion, enforced by the Lagrangian method, to achieve stable, symmetric, and energy-efficient walking for humanoid robots. We evaluated ECO against MPC, standard RL with reward shaping, and four state-of-the-art constrained RL methods. Experiments, including sim-to-sim and sim-to-real transfers on the kid-sized humanoid robot BRUCE, demonstrate that ECO significantly reduces energy consumption compared to baselines while maintaining robust walking performance. These results highlight a substantial advancement in energy-efficient humanoid locomotion. All experimental demonstrations can be found on the project website: https://sites.google.com/view/eco-humanoid.