Ruby协作：利用记忆改进自动红队搜索的质量多样性

Ruby Teaming: Improving Quality Diversity Search with Memory for Automated Red Teaming

June 17, 2024

作者: Vernon Toh Yan Han, Rishabh Bhardwaj, Soujanya Poria

cs.AI

摘要

我们提出了Ruby Teaming 方法，通过将内存缓存作为第三维度，改进了Rainbow Teaming。内存维度为变异器提供线索，以产生更高质量的提示，无论是在攻击成功率（ASR）还是质量多样性方面。Ruby Teaming 生成的提示存档具有74% 的 ASR，比基准线高出20%。在质量多样性方面，Ruby Teaming 在Shannon's Evenness Index（SEI）和Simpson's Diversity Index（SDI）上分别比 Rainbow Teaming 高出6% 和 3%。

English

We propose Ruby Teaming, a method that improves on Rainbow Teaming by including a memory cache as its third dimension. The memory dimension provides cues to the mutator to yield better-quality prompts, both in terms of attack success rate (ASR) and quality diversity. The prompt archive generated by Ruby Teaming has an ASR of 74%, which is 20% higher than the baseline. In terms of quality diversity, Ruby Teaming outperforms Rainbow Teaming by 6% and 3% on Shannon's Evenness Index (SEI) and Simpson's Diversity Index (SDI), respectively.