在上下文中演化提示：一种开放式、自我复制的视角

Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective

June 22, 2025

作者: Jianyu Wang, Zhiqiang Hu, Lidong Bing

cs.AI

摘要

我们提出了一种新颖的提示设计范式，挑战了大型语言模型（LLM）提示中的传统观念。尽管传统智慧强调精心设计的指令和示例对于上下文学习（ICL）的重要性，但我们发现，将随机示例修剪成看似不连贯的“胡言乱语”反而能显著提升多种任务的表现。值得注意的是，这种“胡言乱语”始终匹配或超越了最先进的自动提示优化技术，无论LLM是否经过对齐，都能实现显著增益。然而，发现一种有效的修剪策略并非易事，现有的归因方法和提示压缩算法均未能提供稳健的结果，更不用说依赖人类直觉了。针对这一点，我们提出了一个自我发现的提示优化框架——PromptQuine，这是一个进化搜索框架，仅利用少量数据自动搜索修剪策略。正如自然界中因资源限制而涌现的复杂性——如共生与自组织——我们的框架通过仅利用上下文中的标记，进化并提炼出非传统但极为有效的提示。我们展示了该框架在分类、多选问答、生成及数学推理任务中的有效性，同时保持了良好的运行时效率。我们希望我们的发现能引导对上下文学习的机制研究，并呼吁采取行动，为开发更开放式的搜索算法铺平道路，以实现更有效的LLM提示。

English

We propose a novel prompt design paradigm that challenges conventional wisdom in large language model (LLM) prompting. While conventional wisdom prioritizes well-crafted instructions and demonstrations for in-context learning (ICL), we show that pruning random demonstrations into seemingly incoherent "gibberish" can remarkably improve performance across diverse tasks. Notably, the "gibberish" always matches or surpasses state-of-the-art automatic prompt optimization techniques, achieving substantial gains regardless of LLM alignment. Nevertheless, discovering an effective pruning strategy is non-trivial, as existing attribution methods and prompt compression algorithms fail to deliver robust results, let alone human intuition. In terms of this, we propose a self-discover prompt optimization framework, PromptQuine, an evolutionary search framework that automatically searches for the pruning strategy by itself using only low-data regimes. Much like the emergent complexity in nature--such as symbiosis and self-organization--arising in response to resource constraints, our framework evolves and refines unconventional yet highly effective prompts by leveraging only the tokens present within the context. We demonstrate its effectiveness across classification, multi-choice question answering, generation and math reasoning tasks across LLMs, while achieving decent runtime efficiency. We hope our findings can guide mechanistic studies on in-context learning, and provide a call to action, to pave the way for more open-ended search algorithms for more effective LLM prompting.