TROPT：一個用於統一和推進離散文本優化的開放框架

摘要

離散文本觸發優化——搜尋能夠引導模型朝向特定目標的文本序列——支撐著模型紅隊測試（例如大語言模型越獄）以及審計與可解釋性工作。然而，當前離散優化器的發展與應用面臨阻礙。首先，現有優化器即便開源，也散落在與特定模型、目標及問題領域綁定的研究代碼庫中。其次，優化器變體層出不窮，每個變體都需要額外的工程成本才能使用或擴展，且難以進行直接比較。這些因素共同提高了在現有或新領域採用優化器的門檻，也阻礙了透過新策略推動其進步。為解決這些問題，我們提出了TROPT——首個將離散優化器的執行統一化，並在單一介面下標準化其開發流程的開源框架。TROPT允許使用者透過更換任何組件（模型、目標與優化器）輕鬆自訂端到端優化配方，從而將應用範圍擴展至多個領域與新場景。目前，TROPT內建超過30種優化配方（涵蓋越獄與探測模型內部結構等應用），這些配方由15種以上的優化器（從白箱到黑箱訪問模式）以及15種以上的損失函數（從基礎方法到最先進技術）組合而成。為展示其實用性，我們利用TROPT進行了多項研究：（i）在控制條件下進行大規模實驗，比較並改進針對大語言模型越獄的優化策略，從而揭示出強效但未獲廣泛採用的技術；（ii）將優化器從一個領域（如大語言模型越獄）移植到新領域（例如用於語料庫投毒的嵌入模型）。總而言之，TROPT顯著降低了採用與推動離散文本優化技術的門檻。

English

Discrete text-trigger optimization -- searching for text sequences that, when ingested by a model, steer it toward a specified objective -- underpins model red-teaming (e.g., LLM jailbreaks), as well as auditing and interpretability. However, the current state of discrete optimizers hinders their adoption and progress. First, existing optimizers, when open-sourced at all, are scattered across research codebases tied to specific models, objectives, and problem domains. Second, optimizer variants proliferate, each requiring engineering overhead to use or extend, and remaining hard to compare head-to-head. Together, these raise the bar for adopting optimizers in existing or new domains, and for advancing them via new strategies. We address these gaps with TROPT, the first open-source framework that unifies discrete optimizers' execution and standardizes their development under a single interface. TROPT makes it easy to customize end-to-end optimization recipes by swapping any component -- models, objectives, and optimizers -- extending its reach across domains and new applications. TROPT currently ships with 30+ optimization recipes -- covering applications such as jailbreaking and probing model internals -- built from 15+ optimizers (spanning white-box to black-box access) and 15+ losses, from foundational to state-of-the-art methods. Demonstrating its utility, we leverage TROPT in several studies: (i) controlled, large-scale experiments comparing and enhancing optimization strategies for LLM jailbreaks, revealing potent-yet-underadopted techniques; and (ii) porting optimizers from one domain (e.g., LLM jailbreak) to new domains (e.g., corpus-poisoning embedding model). In all, TROPT significantly lowers the barrier to adopting and advancing discrete text optimization.