大规模法律嵌入基准（MLEB）

摘要

我们推出大规模法律嵌入基准测试（MLEB），这是迄今为止规模最大、多样性最丰富且最全面的开源法律信息检索基准。该基准包含十个经专家标注的数据集，涵盖多个司法管辖区（美国、英国、欧盟、澳大利亚、爱尔兰和新加坡）、多种文档类型（案例、法规、监管指南、合同和文献）以及多种任务类型（检索、零样本分类和问答）。为弥补开源法律信息检索领域在司法管辖范围和专业领域上的空白，MLEB中有七个数据集为全新构建。我们详细记录了构建MLEB及创建新组件数据集的方法论，并公开代码、结果与数据，以助力可复现的评估研究。

English

We present the Massive Legal Embedding Benchmark (MLEB), the largest, most diverse, and most comprehensive open-source benchmark for legal information retrieval to date. MLEB consists of ten expert-annotated datasets spanning multiple jurisdictions (the US, UK, EU, Australia, Ireland, and Singapore), document types (cases, legislation, regulatory guidance, contracts, and literature), and task types (search, zero-shot classification, and question answering). Seven of the datasets in MLEB were newly constructed in order to fill domain and jurisdictional gaps in the open-source legal information retrieval landscape. We document our methodology in building MLEB and creating the new constituent datasets, and release our code, results, and data openly to assist with reproducible evaluations.

大规模法律嵌入基准（MLEB）

The Massive Legal Embedding Benchmark (MLEB)

摘要

Support