Redirecting to original paper in 30 seconds...
Click below to go immediately or wait for automatic redirect
📄 Abstract
Abstract: We present the Massive Legal Embedding Benchmark (MLEB), the largest, most
diverse, and most comprehensive open-source benchmark for legal information
retrieval to date. MLEB consists of ten expert-annotated datasets spanning
multiple jurisdictions (the US, UK, EU, Australia, Ireland, and Singapore),
document types (cases, legislation, regulatory guidance, contracts, and
literature), and task types (search, zero-shot classification, and question
answering). Seven of the datasets in MLEB were newly constructed in order to
fill domain and jurisdictional gaps in the open-source legal information
retrieval landscape. We document our methodology in building MLEB and creating
the new constituent datasets, and release our code, results, and data openly to
assist with reproducible evaluations.
Authors (3)
Umar Butler
Abdur-Rahman Butler
Adrian Lucas Malec
Submitted
October 22, 2025
Key Contributions
Presents the Massive Legal Embedding Benchmark (MLEB), the largest and most diverse open-source benchmark for legal information retrieval. MLEB includes ten expert-annotated datasets across multiple jurisdictions and document types, with seven newly constructed datasets to fill existing gaps, facilitating reproducible evaluations of legal NLP models.
Business Value
Accelerates the development and adoption of advanced AI tools for the legal sector by providing a standardized, comprehensive platform for evaluating and comparing legal NLP models.