xbench：透過專業對齊的現實世界評估追蹤代理生產力的擴展

摘要

我們推出xbench，這是一個動態的、與專業領域對齊的評估套件，旨在彌合AI代理能力與現實世界生產力之間的差距。現有的基準測試往往專注於孤立的技術技能，可能無法準確反映代理在專業環境中提供的經濟價值。為解決這一問題，xbench針對具有商業重要性的領域，由行業專業人士定義評估任務。我們的框架創建了與生產力價值高度相關的指標，能夠預測技術市場契合度（TMF），並促進產品能力隨時間的追蹤。作為初步實施，我們展示了兩個基準測試：招聘和營銷。在招聘方面，我們從真實的獵頭業務場景中收集了50項任務，以評估代理在公司映射、信息檢索和人才搜尋方面的能力。在營銷方面，我們評估代理匹配影響者與廣告商需求的能力，使用一個包含836名候選影響者的精選池，對50項廣告商需求進行性能評估。我們展示了當代領先代理的初步評估結果，為這些專業領域建立了基準。我們持續更新的評估集和評估結果可在https://xbench.org獲取。

English

We introduce xbench, a dynamic, profession-aligned evaluation suite designed to bridge the gap between AI agent capabilities and real-world productivity. While existing benchmarks often focus on isolated technical skills, they may not accurately reflect the economic value agents deliver in professional settings. To address this, xbench targets commercially significant domains with evaluation tasks defined by industry professionals. Our framework creates metrics that strongly correlate with productivity value, enables prediction of Technology-Market Fit (TMF), and facilitates tracking of product capabilities over time. As our initial implementations, we present two benchmarks: Recruitment and Marketing. For Recruitment, we collect 50 tasks from real-world headhunting business scenarios to evaluate agents' abilities in company mapping, information retrieval, and talent sourcing. For Marketing, we assess agents' ability to match influencers with advertiser needs, evaluating their performance across 50 advertiser requirements using a curated pool of 836 candidate influencers. We present initial evaluation results for leading contemporary agents, establishing a baseline for these professional domains. Our continuously updated evalsets and evaluations are available at https://xbench.org.

xbench：透過專業對齊的現實世界評估追蹤代理生產力的擴展

xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations

摘要

Support