微调语言模型生成稳定的无机材料作为文本

摘要

我们提出对大型语言模型进行微调，以生成稳定材料。虽然这种做法有些不寻常，但在文本编码的原子数据上微调大型语言模型简单易行，且可靠，约90%的采样结构遵守原子位置和电荷的物理约束。通过从学习的机器学习势和金标准密度泛函理论计算中得出的能量高于晶格能的计算，我们展示了我们最强的模型（经过微调的LLaMA-2 70B）可以生成预测为亚稳态的材料，其速率约为CDVAE的两倍（49% vs 28%）。由于文本提示的固有灵活性，我们的模型可以同时用于无条件生成稳定材料、填充部分结构以及文本条件生成。最后，我们展示了语言模型捕捉晶体结构关键对称性的能力随着模型规模的增加而提高，这表明预训练的大型语言模型的偏见出奇地适合原子数据。

English

We propose fine-tuning large language models for generation of stable materials. While unorthodox, fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable, with around 90% of sampled structures obeying physical constraints on atom positions and charges. Using energy above hull calculations from both learned ML potentials and gold-standard DFT calculations, we show that our strongest model (fine-tuned LLaMA-2 70B) can generate materials predicted to be metastable at about twice the rate (49% vs 28%) of CDVAE, a competing diffusion model. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material, infilling of partial structures and text-conditional generation. Finally, we show that language models' ability to capture key symmetries of crystal structures improves with model scale, suggesting that the biases of pretrained LLMs are surprisingly well-suited for atomistic data.

微调语言模型生成稳定的无机材料作为文本

Fine-Tuned Language Models Generate Stable Inorganic Materials as Text

摘要

Support