人工智慧自我改進的遞迴臨界性

摘要

人工智慧日益被用於研製未來人工智慧系統的研發過程。我們研究此回饋在何種條件下會變成自我放大。我們的模型描述了人工智慧能力成長率如何取決於基線研究生產力、遞迴回饋，以及研究進展難度的遞增。我們推導出一個遞迴再生數 R_AI，用以判定改進在跨開發週期中是被放大還是衰減。此量比較回饋的強度與進一步進展變得更加困難的速率。當 R_AI > 1 時，改進的效應會跨開發週期複合累積，使系統進入自我放大狀態；當 R_AI < 1 時，其效應會隨週期更替而減弱。這一轉變取決於人工智慧研發回饋迴路的結構，而不必然發生在模型能力的某個特定水平。因此，系統可能在加速現象可見之前即進入自我放大狀態；快速進展也可能在沒有自我放大的情況下發生。較高的基線研究生產力可以加速進展，但不會改變系統是否處於自我放大狀態；然而，開發週期的時長會成為放大的限制性時間尺度。研究難度的增加則可終結一段自我放大時期。將模型擴展至多個研究行動者後顯示，跨組織共享的改進可以讓整體研究生態系統進入自我放大狀態，即使沒有任何個別行動者如此。該框架識別出人工智慧研發系統中可測量的屬性，有助於區分遞迴放大與由其他來源驅動的快速進展，這些屬性包括：遞迴回饋的強度、改進有效傳播至後繼系統的程度、週期時長，以及進一步進展所面臨的難度遞增。

English

AI is increasingly used in the R\&D process that produces future AI systems. We study the conditions under which this feedback becomes self-amplifying. Our model describes how the rate of AI capability growth depends on baseline research productivity, recursive feedback, and the increasing difficulty of research progress. We derive a recursive reproduction number, R_{AI}, that determines whether improvements are amplified or damped across development cycles. This quantity compares the strength of feedback with the rate at which further progress becomes more difficult. When R_{AI}>1, the effects of improvements compound across development cycles, placing the system in a self-amplifying regime. When R_{AI}<1, their effects weaken across cycles. The transition depends on the structure of the AI R\&D feedback loop and need not occur at any particular level of model capability. A system can therefore enter a self-amplifying regime before acceleration becomes visible, while rapid progress can also occur without self-amplification. Higher baseline research productivity can accelerate progress without changing whether the system is self-amplifying, but the duration of the development cycle becomes a limiting timescale for amplification. Increasing research difficulty can end a period of self-amplification. Extending the model to multiple research actors shows that improvements shared across organizations can make the overall research ecosystem self-amplifying even when no individual actor is. The framework identifies measurable properties of AI R\&D systems that can help distinguish recursive amplification from rapid progress driven by other sources, including the strength of recursive feedback, how effectively improvements propagate into successor systems, cycle duration, and the increasing difficulty of further progress.