人工智能自改进的递归临界性

摘要

人工智能越来越多地被用于产生未来人工智能系统的研发过程中。我们研究了这种反馈何时会变得自我放大。我们的模型描述了人工智能能力增长速度如何取决于基线研究生产率、递归反馈以及研究进展日益增加的难度。我们推导出一个递归再生数R_{AI}，它决定了改进在开发周期中是被放大还是被衰减。该量将反馈强度与进一步进展变得更加困难的速度进行比较。当R_{AI} > 1时，改进的效应在多个开发周期中叠加累积，使系统进入自我放大状态。当R_{AI} < 1时，其效应在周期中逐渐减弱。这一转变取决于人工智能研发反馈回路的结构，且不必发生在任何特定的模型能力水平上。因此，系统可能在加速变得可见之前就进入自我放大状态，而快速进展也可能在没有自我放大的情况下发生。更高的基线研究生产率可以加速进展，但不会改变系统是否自我放大，然而开发周期的时长成为放大的一个限制性时间尺度。研究难度的增加可以终结一段自我放大时期。将该模型扩展到多个研究主体表明，跨组织共享的改进可以使整个研究生态系统实现自我放大，即使在没有任何单一主体实现自我放大的情况下也是如此。该框架识别出人工智能研发系统的可测量属性，有助于将递归放大与由其他来源驱动的快速进展区分开来，这些属性包括递归反馈的强度、改进有效传播到后继系统的程度、周期时长以及进一步进展日益增加的难度。

English

AI is increasingly used in the R\&D process that produces future AI systems. We study the conditions under which this feedback becomes self-amplifying. Our model describes how the rate of AI capability growth depends on baseline research productivity, recursive feedback, and the increasing difficulty of research progress. We derive a recursive reproduction number, R_{AI}, that determines whether improvements are amplified or damped across development cycles. This quantity compares the strength of feedback with the rate at which further progress becomes more difficult. When R_{AI}>1, the effects of improvements compound across development cycles, placing the system in a self-amplifying regime. When R_{AI}<1, their effects weaken across cycles. The transition depends on the structure of the AI R\&D feedback loop and need not occur at any particular level of model capability. A system can therefore enter a self-amplifying regime before acceleration becomes visible, while rapid progress can also occur without self-amplification. Higher baseline research productivity can accelerate progress without changing whether the system is self-amplifying, but the duration of the development cycle becomes a limiting timescale for amplification. Increasing research difficulty can end a period of self-amplification. Extending the model to multiple research actors shows that improvements shared across organizations can make the overall research ecosystem self-amplifying even when no individual actor is. The framework identifies measurable properties of AI R\&D systems that can help distinguish recursive amplification from rapid progress driven by other sources, including the strength of recursive feedback, how effectively improvements propagate into successor systems, cycle duration, and the increasing difficulty of further progress.