AI Agent Analysis

harbor-framework/harbor

Framework for evaluating and improving agents

stars

3.7k

Language

Python

Source and compliance noteLast synced: Aug 1, 2026

Git-Stars is independent and not affiliated with GitHub or this project. Analysis may be AI-assisted and based on public repository metadata plus short README-derived summaries. We do not mirror full README files, docs, issues, or social comments.

Original GitHub source Methodology Editorial Policy

Overview

Harbor is a framework for evaluating and optimizing AI agents and language models, enabling developers to run standardized benchmarks, build custom environments, and execute large-scale parallel experiments across multiple cloud providers.

Installation

Install via `uv tool install harbor` or `pip install harbor`.

Problem solved

Harbor solves the fragmentation and reproducibility problem in agent evaluation by providing a unified harness that works with diverse agents (e.g., Claude Code, OpenHands) and benchmarks (e.g., Terminal-Bench, SWE-Bench), while supporting parallel execution and RL rollout generation.

What you can build

Developers can evaluate any agent against existing benchmarks, create and share custom benchmarks, run thousands of parallel environments using providers like Daytona and Modal, and generate rollouts for reinforcement learning optimization. Real-world use cases include benchmarking coding agents, testing model improvements, and scaling agent evaluation for research.

Community sentiment

Positive

No community feedback yet.

Concerns

No concerns documented yet.

Bottom line

Harbor is ideal for researchers and engineers who need a scalable, reproducible way to evaluate and improve agents across multiple benchmarks and environments. It may be overkill for simple single-agent testing, but its parallel execution and provider integration make it powerful for serious evaluation workflows.

Analyzed by Git-Stars - 7/31/2026