Benchmarks
Every number DD publishes comes from a recorded trial whose code and data are in the repository. Read the caveats with the tables: a few projects, a few models, small samples, tasks chosen where knowledge matters.
- Full benchmark: four projects, Opus 5.5 and Haiku 5.5 at high effort (2026-10-18): 26 pre-registered tasks over four projects, 624 runs, every harness fault found on the way listed.
- DD v2 agent trial: Opus 5.5 and Haiku 5.5 at high effort (2026-10-14)
- The first v2 trial (Claude Sonnet 5.5, 2026-10) is summarised in How it is measured.
Full benchmark, Claude Haiku 5.5
2026-10-18 · 26 tasks · 3 runs per condition · judged blind by claude-opus-5-5. Values with their 95% interval where the report has one; changes against agent alone.
| Metric | Agent alone | With documentation | With documentation, not referenced | With DD |
|---|---|---|---|---|
| Tasks resolved | 92% (84–96) | 97% (91–99) +5% | 100% (95–100) +9% | 92% (84–96) +0% |
| Code quality | 85% (81–90) | 97% (95–99) +14% | 97% (95–99) +14% | 94% (91–97) +11% |
| The project's own requirements met | 43% (30–57) | 98% (93–100) +128% | 100% (100–100) +133% | 96% (89–100) +123% |
| Context | 46.1k tokens | 54.8k tokens +19% | 52.6k tokens +14% | 50.1k tokens +9% |
| Cost per task | $0.021 | $0.023 +13% | $0.024 +16% | $0.027 +30% |
Four projects, 26 tasks selected where Haiku alone failed or controls, three runs each; on track, not a promise for yours. Full report
Full benchmark, Claude Opus 5.5
2026-10-18 · 26 tasks · 3 runs per condition · judged blind by claude-opus-5-5. Values with their 95% interval where the report has one; changes against agent alone.
| Metric | Agent alone | With documentation | With documentation, not referenced | With DD |
|---|---|---|---|---|
| Tasks resolved | 95% (88–98) | 100% (95–100) +5% | 99% (93–100) +4% | 100% (95–100) +5% |
| Code quality | 92% (89–95) | 97% (95–99) +5% | 98% (95–100) +7% | 97% (95–99) +5% |
| The project's own requirements met | 69% (56–82) | 100% (100–100) +45% | 98% (93–100) +42% | 100% (100–100) +45% |
| Context | 37.8k tokens | 41.3k tokens +9% | 40.4k tokens +7% | 39.7k tokens +5% |
| Cost per task | $0.428 | $0.501 +17% | $0.474 +11% | $0.455 +6% |
Four projects, 26 tasks selected where Haiku alone failed or controls, three runs each; on track, not a promise for yours. Full report
Trial v2, Claude Haiku 5.5
2026-10-14 · 10 tasks · 1 run per condition · judged blind by claude-opus-5-5. Values with their 95% interval where the report has one; changes against agent alone.
| Metric | Agent alone | With documentation | With DD |
|---|---|---|---|
| Tasks resolved | 100% (72–100) | 100% (72–100) +0% | 100% (72–100) +0% |
| Code quality | 100% (100–100) | 100% (100–100) +0% | 100% (100–100) +0% |
| The project's own requirements met | 100% (100–100) | 100% (100–100) +0% | 100% (100–100) +0% |
| Context | 45.6k tokens | 46.6k tokens +2% | 46.2k tokens +1% |
| Cost per task | $0.021 | $0.020 -5% | $0.020 -7% |
One project, ten tasks, one run per condition: on track, not a promise for yours. Full report
Trial v2, Claude Opus 5.5
2026-10-14 · 10 tasks · 1 run per condition · judged blind by claude-opus-5-5. Values with their 95% interval where the report has one; changes against agent alone.
| Metric | Agent alone | With documentation | With DD |
|---|---|---|---|
| Tasks resolved | 100% (72–100) | 100% (72–100) +0% | 100% (72–100) +0% |
| Code quality | 96% (91–100) | 98% (94–100) +2% | 100% (100–100) +4% |
| The project's own requirements met | 100% (100–100) | 100% (100–100) +0% | 100% (100–100) +0% |
| Context | 36.2k tokens | 37.1k tokens +2% | 35.9k tokens -1% |
| Cost per task | $0.430 | $0.447 +4% | $0.401 -7% |
One project, ten tasks, one run per condition: on track, not a promise for yours. Full report