Evaluation

frites ships two evaluation harnesses under eval/. pnpm eval runs the value-gate (eval/value-gate.ts), a frites-specific A/B that drives a real claude client against the gateway to ask whether fanning out actually beats a single agent on real coding fixtures, and whether the extra cost is worth it. pnpm bench runs the bench-matrix (eval/bench-matrix.ts), a standard agentic-coding harness run across many frites configs and raw-model baselines on the same tasks, tabling accuracy, cost, and latency so you can compare frites to normal models. Both are metered (every run fans out to live child CLIs), so smoke-test the wiring first. The full setup, Docker sandbox, invocations, and result-reading guidance live in the canonical runbook at ../../eval/README.md.