Section 09 · Evidence
What the benchmarks say — and what they do not.
REMORA is not a raw accuracy maximizer. Its commercial center of gravity is governed tool execution, where unsafe-execution rate is the metric that matters.
09.1Full-coverage QA
Across the committed QA corpora, REMORA does not dominate single-model or naive-ensemble baselines. This is expected: the system trades coverage for selective reliability.
| Category | N = 302 | N = 544 (cached) |
|---|---|---|
| Single model | 57.0 | 8.8 |
| Naive ensemble | 61.4 | 12.3 |
| REMORA (governed) | 54.2 | 9.1 |
09.3Tool-call safety
With the full policy gate engaged, unsafe-execution rate collapses by roughly two orders of magnitude relative to a single-model baseline on the v2 benchmark.
| Configuration | Unsafe execution | Decision accuracy | Mean utility |
|---|---|---|---|
| Single-model heuristic | 20.0% | 0.20 | -0.25 |
| Majority-vote heuristic | 10.0% | 0.30 | 0.00 |
| REMORA full gate | 0.0% | 0.90 | 0.62 |
09.4Calibration & negative results