EBench：通用型移動操作策略的元素級診斷

摘要

我們提出EBench，這是一個模擬基準測試，用於診斷通用型移動操作策略，超越單一的成功率指標。EBench包含26個多樣且具挑戰性的操作任務，並沿5個能力維度與4個泛化維度進行標註。我們評估了最先進的通用操作模型，包括π_0、π_{0.5}、XVLA與InternVLA-A1，並發現成功率相近的模型在能力特徵上存在顯著差異：π_{0.5}達到最高的測試成功率與最佳的訓練-測試保留率，而InternVLA-A1在移動操作上表現突出，但在靈巧任務上則明顯不足；XVLA則在一組與其他策略不重疊的基本技能上展現優勢。除了能力特徵分析，EBench還從4個具代表性的角度探討泛化能力，識別不同分佈偏移因素的影響。結果揭示了各模型在總體分數背後的優缺點。我們期望此基準能提供廣泛的診斷信號，以引導通用操作模型的迭代改進。

English

We present EBench, a simulation benchmark that diagnoses generalist mobile manipulation policies beyond a single success-rate scalar. EBench comprises 26 diverse and challenging manipulation tasks annotated along 5 capability dimensions and 4 generalization dimensions. We evaluate state-of-the-art generalist manipulation models including π_0, π_{0.5}, XVLA, and InternVLA-A1, and reveal that models with near success rates exhibit strikingly different capability profiles: π_{0.5} achieves the highest test success rate and the best train--test retention, whereas InternVLA-A1 dominates mobile manipulation but collapses on dexterous tasks, and XVLA exhibits strengths on a disjoint set of atomic skills compared to other policies. Beyond capability profiling, EBench analyzes the generalization ability from 4 representative perspectives, identifying the impact of different distribution shift factors. The results reveal strengths and weaknesses of models behind an overall score. We hope this benchmark offers a broad set of diagnostic signals to guide iteration on generalist manipulation models.