大规模法律嵌入基准（MLEB）

The Massive Legal Embedding Benchmark (MLEB)

October 22, 2025

作者: Umar Butler, Abdur-Rahman Butler, Adrian Lucas Malec

cs.AI

摘要

我们推出大规模法律嵌入基准测试（MLEB），这是迄今为止规模最大、最多样化且最全面的开源法律信息检索基准。MLEB包含十个经专家标注的数据集，涵盖多个司法管辖区（美国、英国、欧盟、澳大利亚、爱尔兰和新加坡）、多种文档类型（案例、法规、监管指南、合同和文献）以及多种任务类型（检索、零样本分类和问答）。为填补开源法律信息检索领域在专业范畴与司法管辖权方面的空白，MLEB中有七个数据集为全新构建。我们详细记录了构建MLEB及创建新组件数据集的方法论，并公开共享代码、结果和数据，以助力可复现的评估研究。

English

We present the Massive Legal Embedding Benchmark (MLEB), the largest, most diverse, and most comprehensive open-source benchmark for legal information retrieval to date. MLEB consists of ten expert-annotated datasets spanning multiple jurisdictions (the US, UK, EU, Australia, Ireland, and Singapore), document types (cases, legislation, regulatory guidance, contracts, and literature), and task types (search, zero-shot classification, and question answering). Seven of the datasets in MLEB were newly constructed in order to fill domain and jurisdictional gaps in the open-source legal information retrieval landscape. We document our methodology in building MLEB and creating the new constituent datasets, and release our code, results, and data openly to assist with reproducible evaluations.