Industrial-Instruction：用於從工業技術報告建構指令微調與基準資料集的端到端框架

摘要

工業技術報告包含對於維護、故障排除與產品工程具有高價值的知識，但其異質結構（密集的文字敘述、規格、表格）使得標準檢索與問答流程難以對其進行索引和推理，且目前沒有從此類文件建構的公開指令微調或基準資料集。我們透過 Industrial-Instruction 填補此缺口，貢獻 (i) 兩個從真實工業技術報告建構的開放式問答資料集，以及 (ii) 產生這些資料集的端對端流程。利用 906 份公開的 Panasonic 文件（7,525 頁），我們應用版面感知抽取、建立語義檢索索引，並在五種查詢-文件關係（不相關檢索、單文件/多文件支持、單文件/多文件答案）下，綜合以檢索到的證據為基礎的多選題問答。在過濾初始的 23.9k 個生成樣本後，每個資料集提供約 13.6k 個問答對，附有來源文件和一個留出的基準測試分割。對小型開放式大型語言模型（低於 10B 參數）進行微調，可將 Panasonic 基準上的 Set-Match Accuracy 從 28.5% 提升至 42.0%，F1 從 46.6% 提升至 63.5%。我們發布由相同流程產生的兩個平行版本：一個使用開放權重的 Qwen3-30B-A3B-Instruct 模型生成，另一個使用封閉式、基於 API 的 Claude-Opus-4.6 模型生成，從而能夠直接比較開放式資料生成與前沿模型資料生成。Claude-Opus-4.6 資料集產生了更乾淨的原始語料庫和更大的微調增益，但成本約高出兩個數量級。MMLU 評估顯示，在 Claude-Opus-4.6 資料上訓練的模型基本上保留了所有一般知識，而 Qwen 生成資料則表現出雖小但可測量的遺忘效應。總而言之，這些資料集和流程為從真實世界文件建構可擴展的工業基準和訓練資料，提供了一條實用且可重現的途徑。

English

Industrial technical reports contain high-value knowledge for maintenance, troubleshooting, and product engineering, but their heterogeneous structure (dense prose, specifications, tables) makes them difficult to index and reason over with standard retrieval and QA pipelines, and no public instruction-tuning or benchmark datasets are built from such documents. We address this gap with Industrial-Instruction, contributing (i) two open QA datasets built from real industrial technical reports and (ii) the end-to-end pipeline that produces them. Using 906 public Panasonic documents (7,525 pages), we apply layout-aware extraction, build a semantic retrieval index, and synthesize multiple-choice QA grounded in retrieved evidence under five query-document relationships (irrelevant retrieval, single-/multi-document support, single-/multi-document answer). After filtering an initial 23.9k generated samples, each dataset provides approximately 13.6k QA pairs with source documents and a held-out benchmark split. Fine-tuning small open LLMs (under 10B parameters) improves Set-Match Accuracy from 28.5% to 42.0% and F1 from 46.6% to 63.5% on the Panasonic benchmark. We release two parallel versions built by the same pipeline: one generated with the open-weight Qwen3-30B-A3B-Instruct model and one with the closed, API-based Claude-Opus-4.6 model, enabling a direct comparison of open- versus frontier-model data generation. The Claude-Opus-4.6 dataset yields a cleaner raw corpus and larger fine-tuning gains, at roughly two orders of magnitude higher cost. MMLU evaluation shows models trained on the Claude-Opus-4.6 data retain essentially all general knowledge, versus a small but measurable forgetting effect for the Qwen-generated data. Together, these datasets and pipeline offer a practical, reproducible path toward scalable industrial benchmarks and training data from real-world documentation.