離散時間混合自動機學習：足式運動與滑板運動的交匯

Discrete-Time Hybrid Automata Learning: Legged Locomotion Meets Skateboarding

March 3, 2025

作者: Hang Liu, Sangli Teng, Ben Liu, Wei Zhang, Maani Ghaffari

cs.AI

摘要

本文介紹了離散時間混合自動機學習（DHAL）框架，該框架利用在線強化學習來識別和執行模式切換，而無需進行軌跡分割或事件函數學習。混合動力系統包含連續流動和離散模式切換，能夠模擬如腿式機器人運動等機器人任務。基於模型的方法通常依賴於預定義的步態，而無模型方法則缺乏明確的模式切換知識。現有方法通過分割來識別離散模式，然後回歸連續流動，但在沒有軌跡標籤或分割的情況下學習高維複雜剛體動力學仍是一個具有挑戰性的開放性問題。我們的方法結合了貝塔策略分佈和多評論家架構，以模擬接觸引導的運動，並以具有挑戰性的四足機器人滑板任務為例。我們通過模擬和實際測試驗證了我們的方法，展示了其在混合動力系統中的穩健性能。

English

This paper introduces Discrete-time Hybrid Automata Learning (DHAL), a framework using on-policy Reinforcement Learning to identify and execute mode-switching without trajectory segmentation or event function learning. Hybrid dynamical systems, which include continuous flow and discrete mode switching, can model robotics tasks like legged robot locomotion. Model-based methods usually depend on predefined gaits, while model-free approaches lack explicit mode-switching knowledge. Current methods identify discrete modes via segmentation before regressing continuous flow, but learning high-dimensional complex rigid body dynamics without trajectory labels or segmentation is a challenging open problem. Our approach incorporates a beta policy distribution and a multi-critic architecture to model contact-guided motions, exemplified by a challenging quadrupedal robot skateboard task. We validate our method through simulations and real-world tests, demonstrating robust performance in hybrid dynamical systems.