harveyai/harvey-labs

It addresses the lack of standardized, realistic benchmarks for legal AI agents, which often rely on generic QA datasets or proprietary evaluations. By providing a domain-specific, task-based benchmark with rubrics and an execution harness, it enables objective, reproducible assessment of agent performance on complex legal workflows, facilitating targeted improvements and fair comparisons across models and frameworks.

Developers can build and evaluate AI agents that perform legal tasks such as M&A data-room analysis, contract review, legal research, and drafting, using the provided tasks and harness. The benchmark supports custom task creation, model adapters, and evaluation sweeps, allowing teams to measure agent performance, identify weaknesses, and iterate on agent designs. The ceiling includes achieving human-level or superhuman performance on specific legal workflows, with the potential to drive adoption of AI in legal practice by demonstrating reliability and quality.