Coding agent benchmarks
Pilot: 6 tasks × 1 trial(s). Directional results, not a stable overall ranking.
Completed runs from a six-task local pilot, one trial per task. Other campaign runs remain in progress. This is not an official full Terminal-Bench score or a stable overall model ranking. Timing comparisons are withheld because instrumentation and operational interruptions differed.
| Harness | Model | Provider | Tasks resolved | Successful tasks / USD | Estimated API cost |
|---|---|---|---|---|---|
| Claude Code | Claude Sonnet 5.5 | Anthropic | 1 / 6 | 1.21 | $0.8244 |
| Codex | GPT-6.1 Sol | OpenAI | 4 / 6 | 3.82 | $1.0460 |
| Kimi Code | Kimi K3 | Moonshot | 2 / 6 | 0.25 | $7.9117 |
| Pi | Claude Sonnet 5.5 | Anthropic | 1 / 6 | 1.64 | $0.6108 |
| Pi | GPT-6.1 Sol | OpenAI | 2 / 6 | 3.13 | $0.6399 |
API estimates include failed attempts and retries; compute is excluded. Different cohorts are never combined. Missing or incomparable timing is not ranked.