Ruby 隊伍：透過記憶體來提升自動化紅隊行動的品質多樣性搜索

Ruby Teaming: Improving Quality Diversity Search with Memory for Automated Red Teaming

June 17, 2024

作者: Vernon Toh Yan Han, Rishabh Bhardwaj, Soujanya Poria

cs.AI

摘要

我們提出了Ruby Teaming方法，通過將記憶快取作為其第三維度，改進了Rainbow Teaming。記憶維度為變異器提供提示，以產生更高質量的提示，無論是在攻擊成功率（ASR）還是質量多樣性方面。Ruby Teaming生成的提示存檔具有74%的ASR，比基準線高出20%。就質量多樣性而言，Ruby Teaming在Shannon's Evenness Index（SEI）和Simpson's Diversity Index（SDI）上分別比Rainbow Teaming高出6%和3%。

English

We propose Ruby Teaming, a method that improves on Rainbow Teaming by including a memory cache as its third dimension. The memory dimension provides cues to the mutator to yield better-quality prompts, both in terms of attack success rate (ASR) and quality diversity. The prompt archive generated by Ruby Teaming has an ASR of 74%, which is 20% higher than the baseline. In terms of quality diversity, Ruby Teaming outperforms Rainbow Teaming by 6% and 3% on Shannon's Evenness Index (SEI) and Simpson's Diversity Index (SDI), respectively.