Industrial-Instruction：一种从工业技术报告构建指令微调与基准数据集的端到端框架

摘要

工业技术报告蕴含着面向维护、故障诊断和产品工程的高价值知识，但其异构结构（密集的散文式叙述、规格说明、表格）使得标准检索与问答流水线难以对其进行索引和推理，且目前尚无基于此类文档构建的公开指令微调数据集或基准数据集。我们通过Industrial-Instruction填补了这一空白，贡献了(i)两个基于真实工业技术报告构建的开放问答数据集，以及(ii)生成这些数据集的端到端流水线。利用906份公开的松下文档（共7,525页），我们应用布局感知抽取技术，构建语义检索索引，并在五种查询-文档关系（无关检索、单/多文档支持、单/多文档答案）下，基于检索到的证据合成多项选择问答对。在对初始生成的23.9k条样本进行过滤后，每个数据集提供约13.6k个带源文档的问答对，并包含一个留出的基准测试划分。对小型开源大语言模型（参数低于100亿）进行微调后，在松下基准上的集合匹配准确率从28.5%提升至42.0%，F1分数从46.6%提升至63.5%。我们发布了由同一流水线生成的两个并行版本：一个使用开放权重的Qwen3-30B-A3B-Instruct模型生成，另一个使用基于API的闭源模型Claude-Opus-4.6生成，从而能够直接比较开放模型与前沿模型在数据生成方面的差异。Claude-Opus-4.6数据集产生了更干净的原始语料和更大的微调收益，但成本高出约两个数量级。MMLU评估显示，使用Claude-Opus-4.6数据训练的模型几乎完全保留了一般知识，而使用Qwen生成数据的模型则出现了虽小但可测的遗忘效应。综合而言，这些数据集和流水线为从真实世界文档构建可扩展的工业基准和训练数据提供了一条实用且可复现的路径。

English

Industrial technical reports contain high-value knowledge for maintenance, troubleshooting, and product engineering, but their heterogeneous structure (dense prose, specifications, tables) makes them difficult to index and reason over with standard retrieval and QA pipelines, and no public instruction-tuning or benchmark datasets are built from such documents. We address this gap with Industrial-Instruction, contributing (i) two open QA datasets built from real industrial technical reports and (ii) the end-to-end pipeline that produces them. Using 906 public Panasonic documents (7,525 pages), we apply layout-aware extraction, build a semantic retrieval index, and synthesize multiple-choice QA grounded in retrieved evidence under five query-document relationships (irrelevant retrieval, single-/multi-document support, single-/multi-document answer). After filtering an initial 23.9k generated samples, each dataset provides approximately 13.6k QA pairs with source documents and a held-out benchmark split. Fine-tuning small open LLMs (under 10B parameters) improves Set-Match Accuracy from 28.5% to 42.0% and F1 from 46.6% to 63.5% on the Panasonic benchmark. We release two parallel versions built by the same pipeline: one generated with the open-weight Qwen3-30B-A3B-Instruct model and one with the closed, API-based Claude-Opus-4.6 model, enabling a direct comparison of open- versus frontier-model data generation. The Claude-Opus-4.6 dataset yields a cleaner raw corpus and larger fine-tuning gains, at roughly two orders of magnitude higher cost. MMLU evaluation shows models trained on the Claude-Opus-4.6 data retain essentially all general knowledge, versus a small but measurable forgetting effect for the Qwen-generated data. Together, these datasets and pipeline offer a practical, reproducible path toward scalable industrial benchmarks and training data from real-world documentation.