超越語言模型：位元模型是數位世界模擬器。

摘要

傳統的深度學習常常忽略了位元組，這是數位世界中的基本單位，所有形式的資訊和操作都是以二進制格式編碼和操作的。受自然語言處理中下一個標記預測成功的啟發，我們引入了bGPT，一個具有下一個位元組預測的模型，以模擬數位世界。bGPT在各種模式下，包括文本、音訊和影像方面的表現與專門化模型相匹敵，並為預測、模擬和診斷演算法或硬體行為提供了新的可能性。它幾乎完美地複製了將符號音樂資料轉換的過程，將ABC樂譜轉換為MIDI格式的錯誤率僅為每位元組0.0011位元。此外，bGPT在模擬CPU行為方面展現出卓越的能力，執行各種操作的準確率超過99.99%。利用下一個位元組預測，像bGPT這樣的模型可以直接從龐大的二進制資料中學習，有效地模擬數位世界的複雜模式。

English

Traditional deep learning often overlooks bytes, the basic units of the digital world, where all forms of information and operations are encoded and manipulated in binary format. Inspired by the success of next token prediction in natural language processing, we introduce bGPT, a model with next byte prediction to simulate the digital world. bGPT matches specialized models in performance across various modalities, including text, audio, and images, and offers new possibilities for predicting, simulating, and diagnosing algorithm or hardware behaviour. It has almost flawlessly replicated the process of converting symbolic music data, achieving a low error rate of 0.0011 bits per byte in converting ABC notation to MIDI format. In addition, bGPT demonstrates exceptional capabilities in simulating CPU behaviour, with an accuracy exceeding 99.99% in executing various operations. Leveraging next byte prediction, models like bGPT can directly learn from vast binary data, effectively simulating the intricate patterns of the digital world.

超越語言模型：位元模型是數位世界模擬器。

Beyond Language Models: Byte Models are Digital World Simulators

摘要

Support