演化中的上下文提示：一種開放式、自我複製的視角

Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective

June 22, 2025

作者: Jianyu Wang, Zhiqiang Hu, Lidong Bing

cs.AI

摘要

我們提出了一種新穎的提示設計範式，挑戰了大型語言模型（LLM）提示中的傳統智慧。傳統智慧強調精心設計的指令和示範以實現上下文學習（ICL），而我們則展示，將隨機示範修剪成看似不連貫的“胡言亂語”能夠顯著提升多樣任務的表現。值得注意的是，這種“胡言亂語”始終匹配或超越最先進的自動提示優化技術，無論LLM對齊與否，均能取得顯著增益。然而，發現有效的修剪策略並非易事，現有的歸因方法和提示壓縮算法均未能提供穩健的結果，更不用說人類直覺了。針對這一點，我們提出了一種自我發現的提示優化框架——PromptQuine，這是一種進化搜索框架，僅利用低數據量即可自動搜索修剪策略。正如自然界中因資源限制而湧現的複雜性——如共生與自組織——我們的框架通過僅利用上下文中的詞彙，進化並精煉出非常規但極其有效的提示。我們在LLM的分類、多選題回答、生成及數學推理任務中展示了其有效性，同時保持了良好的運行效率。我們希望我們的發現能引導對上下文學習的機制研究，並發出行動號召，為開發更開放式的搜索算法以實現更有效的LLM提示鋪平道路。

English

We propose a novel prompt design paradigm that challenges conventional wisdom in large language model (LLM) prompting. While conventional wisdom prioritizes well-crafted instructions and demonstrations for in-context learning (ICL), we show that pruning random demonstrations into seemingly incoherent "gibberish" can remarkably improve performance across diverse tasks. Notably, the "gibberish" always matches or surpasses state-of-the-art automatic prompt optimization techniques, achieving substantial gains regardless of LLM alignment. Nevertheless, discovering an effective pruning strategy is non-trivial, as existing attribution methods and prompt compression algorithms fail to deliver robust results, let alone human intuition. In terms of this, we propose a self-discover prompt optimization framework, PromptQuine, an evolutionary search framework that automatically searches for the pruning strategy by itself using only low-data regimes. Much like the emergent complexity in nature--such as symbiosis and self-organization--arising in response to resource constraints, our framework evolves and refines unconventional yet highly effective prompts by leveraging only the tokens present within the context. We demonstrate its effectiveness across classification, multi-choice question answering, generation and math reasoning tasks across LLMs, while achieving decent runtime efficiency. We hope our findings can guide mechanistic studies on in-context learning, and provide a call to action, to pave the way for more open-ended search algorithms for more effective LLM prompting.