SkillHarness：為電腦使用代理駕馭安全技能

摘要

電腦使用代理（CUA）日益部署於動態互動環境中，這使得在互動過程中持續學習技能的需求與日俱增。近期研究透過從成功軌跡中學習可重複使用的技能來應對此挑戰。然而，這些技能學習方法大多假設處於靜態且安全的環境，忽略了對抗性互動（例如提示注入）與環境動態變化（例如彈出視窗）的風險。在動態環境中，此類假設可能導致高風險的技能學習與脆弱的執行過程，削弱CUA的可靠性。這引發一個問題：CUA如何在動態環境中安全地學習與使用技能？為解決此問題，我們提出SkillHarness，一個在動態環境中安全運用技能的框架。SkillHarness超越靜態的技能抽象，將技能學習與使用建模為一項安全受限的互動過程。具體而言，我們引入技能邊界，利用多重監督信號從互動軌跡中識別安全技能，並在技能生命週期中建構自我改善的安全限制。此外，SkillHarness引入選擇性技能重複使用，引導任務根據情境進行分解，並透過選擇性啟動技能子集來完成任務。我們的實驗顯示，SkillHarness將所學技能的不安全率顯著降低57.1%，並在動態環境變化下持續提升執行穩定性，優於現有基準方法。

English

Computer-Use Agents (CUAs) are increasingly deployed in dynamic interactive environments, creating a growing need for continual skill learning during interaction. Recent approaches address this challenge by learning reusable skills from successful trajectories. However, these skill learning methods largely assume static and safe environments, overlooking risks from adversarial interactions (e.g., prompt injections) and environmental dynamics (e.g., pop-ups). In dynamic settings, such assumptions can lead to risky skill learning and brittle execution, undermining the reliability of CUAs. This raises the question: how can CUAs learn and use skills safely in dynamic environments? To address this problem, we propose SkillHarness, a framework for safe skill harnessing in dynamic environments. SkillHarness moves beyond static skill abstractions by modeling skill learning and utilization as a safety-constrained interaction process. Specifically, we introduce the skill boundary that leverages multi-source supervision signals to identify safe skills from interaction trajectories, and construct self-improving safety constraints throughout the skill lifecycle. In addition, SkillHarness introduces selective skill reuse, where tasks are guided to decompose according to context and completed through the selective activation of skill subsets. Our experiments demonstrate that SkillHarness significantly reduces the unsafe rate of learned skills by 57.1% and consistently improves execution stability under dynamic environmental changes, outperforming existing baselines.