Rapporto Tecnico di InternLM2
InternLM2 Technical Report
March 26, 2024
Autori: Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, Shuaibin Li, Wei Li, Yining Li, Hongwei Liu, Jiangning Liu, Jiawei Hong, Kaiwen Liu, Kuikun Liu, Xiaoran Liu, Chengqi Lv, Haijun Lv, Kai Lv, Li Ma, Runyuan Ma, Zerun Ma, Wenchang Ning, Linke Ouyang, Jiantao Qiu, Yuan Qu, Fukai Shang, Yunfan Shao, Demin Song, Zifan Song, Zhihao Sui, Peng Sun, Yu Sun, Huanze Tang, Bin Wang, Guoteng Wang, Jiaqi Wang, Jiayu Wang, Rui Wang, Yudong Wang, Ziyi Wang, Xingjian Wei, Qizhen Weng, Fan Wu, Yingtong Xiong, Chao Xu, Ruiliang Xu, Hang Yan, Yirong Yan, Xiaogui Yang, Haochen Ye, Huaiyuan Ying, Jia Yu, Jing Yu, Yuhang Zang, Chuyu Zhang, Li Zhang, Pan Zhang, Peng Zhang, Ruijie Zhang, Shuo Zhang, Songyang Zhang, Wenjian Zhang, Wenwei Zhang, Xingcheng Zhang, Xinyue Zhang, Hui Zhao, Qian Zhao, Xiaomeng Zhao, Fengzhe Zhou, Zaida Zhou, Jingming Zhuo, Yicheng Zou, Xipeng Qiu, Yu Qiao, Dahua Lin
cs.AI
Abstract
L'evoluzione dei Large Language Models (LLM) come ChatGPT e GPT-4 ha stimolato discussioni sull'avvento dell'Intelligenza Artificiale Generale (AGI). Tuttavia, replicare tali progressi in modelli open-source si è rivelato impegnativo. Questo articolo introduce InternLM2, un LLM open-source che supera i suoi predecessori in valutazioni complete su 6 dimensioni e 30 benchmark, nella modellazione di contesti lunghi e in valutazioni soggettive aperte, grazie a innovative tecniche di pre-training e ottimizzazione. Il processo di pre-training di InternLM2 è descritto in dettaglio, evidenziando la preparazione di diversi tipi di dati, inclusi testo, codice e dati a contesto lungo. InternLM2 cattura in modo efficiente le dipendenze a lungo termine, inizialmente addestrato su 4k token prima di progredire a 32k token nelle fasi di pre-training e fine-tuning, dimostrando prestazioni notevoli nel test "Needle-in-a-Haystack" da 200k token. InternLM2 è ulteriormente allineato utilizzando il Supervised Fine-Tuning (SFT) e una nuova strategia di Conditional Online Reinforcement Learning from Human Feedback (COOL RLHF) che affronta preferenze umane conflittuali e il fenomeno del reward hacking. Rilasciando modelli InternLM2 in diverse fasi di addestramento e dimensioni, forniamo alla comunità approfondimenti sull'evoluzione del modello.
English
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has
sparked discussions on the advent of Artificial General Intelligence (AGI).
However, replicating such advancements in open-source models has been
challenging. This paper introduces InternLM2, an open-source LLM that
outperforms its predecessors in comprehensive evaluations across 6 dimensions
and 30 benchmarks, long-context modeling, and open-ended subjective evaluations
through innovative pre-training and optimization techniques. The pre-training
process of InternLM2 is meticulously detailed, highlighting the preparation of
diverse data types including text, code, and long-context data. InternLM2
efficiently captures long-term dependencies, initially trained on 4k tokens
before advancing to 32k tokens in pre-training and fine-tuning stages,
exhibiting remarkable performance on the 200k ``Needle-in-a-Haystack" test.
InternLM2 is further aligned using Supervised Fine-Tuning (SFT) and a novel
Conditional Online Reinforcement Learning from Human Feedback (COOL RLHF)
strategy that addresses conflicting human preferences and reward hacking. By
releasing InternLM2 models in different training stages and model sizes, we
provide the community with insights into the model's evolution.