Harvey BigLaw Bench
- Tests
- Harvey's set of real-world legal tasks for transactional and litigation work. Expert rubrics grade model answers and the sources they cite.
- Best for
- Legal AI market context
- Classification
- scoreboard · current · commercial
AI evaluation domain · 11 source-backed records
Legal AI Benchmarks
Legal benchmarks show limits in reasoning, retrieval, drafting, and document review. Results depend on the jurisdiction and the task. Use public sets to make a shortlist. Then test with your own authorities and workflows.
Decision guidance
How should legal teams use public AI benchmarks?
Check the jurisdiction, task definitions, source dates, and grading method before you read a score. Build private tests for citations, deadlines, forms, privilege, abstention, and the work product your lawyers review.
11
Benchmarks in this domain
Sources first · alphabetical collection order
Runnability describes access to a usable repository, dataset, or harness—not whether setup is easy.
2026-10-26
Harvey Legal Agent Benchmark (LAB)
- Tests
- Tests AI agents on long legal tasks across 24 practice areas. Expert-written rubrics with binary pass or fail criteria grade each task.
- Best for
- Agentic legal work product
- Classification
- agent benchmark · current · open
LawBench
- Tests
- Tests Chinese legal knowledge at three levels: memorize, understand, and apply. Uses 20 tasks of five types, such as classification, extraction, and generation.
- Best for
- Chinese/legal reasoning research
- Classification
- benchmark · current · open
Legal RAG Bench
- Tests
- Tests end-to-end legal RAG (retrieval-augmented generation) systems on 100 expert-written questions about Victorian (Australia) criminal law. Scores retrieval and answer quality separately.
- Best for
- Comparing legal RAG systems where sources and reasoning both matter.
- Classification
- benchmark · emerging · open
LegalBench
- Tests
- Tests legal reasoning with 162 tasks that lawyers and legal experts designed. Covers six types of legal reasoning, such as issue spotting, rule recall, and interpretation.
- Best for
- Legal reasoning baseline
- Classification
- benchmark · current · open
LegalBench-RAG
- Tests
- Tests the retrieval step of legal RAG (retrieval-augmented generation) pipelines. The system must find the exact short text spans in legal documents that answer each query.
- Best for
- Legal document retrieval and grounded answers
- Classification
- benchmark · current · open
LEXam
- Tests
- Tests legal reasoning on 7,537 questions from 340 law exams at Swiss universities, in English and German. Has open long-answer questions and multiple-choice questions.
- Best for
- Comparing legal knowledge across European and international jurisdictions.
- Classification
- benchmark · emerging · open
LexGLUE
- Tests
- Tests legal language understanding in English with seven existing datasets. Tasks include classification of court cases, EU laws, contract clauses, and terms of service.
- Best for
- Classic legal NLP tasks
- Classification
- benchmark · legacy · open
PLawBench
- Tests
- Tests LLMs (large language models) on legal practice: client consultation, case analysis, and drafting of legal documents. Experts write a scoring rubric for each question.
- Best for
- Evaluating legal-practice outputs rather than multiple-choice legal knowledge.
- Classification
- benchmark · emerging · open
RedlineBench
- Tests
- Tests whether AI agents can redline a contract over several negotiation turns. The agent edits a Word file with tracked changes and comments, and attorney rubrics grade it.
- Best for
- Evaluating legal agents that must preserve objectives across revisions.
- Classification
- benchmark · emerging · open
Vals AI LegalBench
- Tests
- A leaderboard where Vals AI runs current models on the LegalBench legal reasoning tasks. It also shows cost and latency per model.
- Best for
- Current legal model comparison
- Classification
- scoreboard · current · commercial