AutoMat：透過代理工具實現顯微鏡下晶體結構自動重建

AutoMat: Enabling Automated Crystal Structure Reconstruction from Microscopy via Agentic Tool Use

May 19, 2025

作者: Yaotian Yang, Yiwen Tang, Yizhe Chen, Xiao Chen, Jiangjie Qiu, Hao Xiong, Haoyu Yin, Zhiyao Luo, Yifei Zhang, Sijia Tao, Wentao Li, Qinghua Zhang, Yuqiang Li, Wanli Ouyang, Bin Zhao, Xiaonan Wang, Fei Wei

cs.AI

摘要

基於機器學習的原子間勢和力場精確依賴於準確的原子結構，然而由於實驗解析晶體的有限性，此類數據稀缺。儘管原子分辨率電子顯微鏡提供了結構數據的潛在來源，但將這些圖像轉換為模擬就緒的格式仍然耗時且易出錯，為模型訓練和驗證造成了瓶頸。我們介紹了AutoMat，這是一個端到端、代理輔助的流程，能夠自動將掃描透射電子顯微鏡（STEM）圖像轉化為原子晶體結構並預測其物理性質。AutoMat結合了模式適應性去噪、物理引導模板檢索、對稱性感知原子重建、快速弛豫及通過MatterSim進行的性質預測，以及所有階段的協調編排。我們為此任務提出了首個專用的STEM2Mat-Bench，並使用晶格均方根偏差（RMSD）、形成能平均絕對誤差（MAE）和結構匹配成功率來評估性能。通過協調外部工具調用，AutoMat使僅基於文本的大型語言模型（LLM）在此領域超越了視覺語言模型，實現了整個流程的閉環推理。在超過450個結構樣本的大規模實驗中，AutoMat顯著優於現有的多模態大型語言模型和工具。這些結果驗證了AutoMat和STEM2Mat-Bench，標誌著在材料科學中橋接顯微鏡與原子級模擬的關鍵一步。代碼和數據集公開於https://github.com/yyt-2378/AutoMat和https://huggingface.co/datasets/yaotianvector/STEM2Mat。

English

Machine learning-based interatomic potentials and force fields depend critically on accurate atomic structures, yet such data are scarce due to the limited availability of experimentally resolved crystals. Although atomic-resolution electron microscopy offers a potential source of structural data, converting these images into simulation-ready formats remains labor-intensive and error-prone, creating a bottleneck for model training and validation. We introduce AutoMat, an end-to-end, agent-assisted pipeline that automatically transforms scanning transmission electron microscopy (STEM) images into atomic crystal structures and predicts their physical properties. AutoMat combines pattern-adaptive denoising, physics-guided template retrieval, symmetry-aware atomic reconstruction, fast relaxation and property prediction via MatterSim, and coordinated orchestration across all stages. We propose the first dedicated STEM2Mat-Bench for this task and evaluate performance using lattice RMSD, formation energy MAE, and structure-matching success rate. By orchestrating external tool calls, AutoMat enables a text-only LLM to outperform vision-language models in this domain, achieving closed-loop reasoning throughout the pipeline. In large-scale experiments over 450 structure samples, AutoMat substantially outperforms existing multimodal large language models and tools. These results validate both AutoMat and STEM2Mat-Bench, marking a key step toward bridging microscopy and atomistic simulation in materials science.The code and dataset are publicly available at https://github.com/yyt-2378/AutoMat and https://huggingface.co/datasets/yaotianvector/STEM2Mat.