# Executable Eval Harness

This harness runs generated two-channel audit eval cases through deterministic offline adapters.
It is designed for local validation of the result-row contract before replacing adapters with a real target runtime.

## Adapter Boundary

- Model adapter: `safe-offline`
- Defense adapter: `channel-isolation`
- Result rows: `examples/eval-harness-results.sample.jsonl`
- Inputs: `dist/audit-eval-cases.jsonl`
- Output contract: JSONL rows accepted by `scripts/summarize_eval_results.py`.
- Each row includes prompt provenance: case id, scenario id, prompt channel, untrusted channel, safe input, and evidence refs.

## Built-In Model Adapters

| Adapter | Verdict shape | Use |
|---|---|---|
| `safe-offline` | pass | Demonstrates expected resilient behavior. |
| `vulnerable-offline` | fail | Demonstrates expected vulnerable behavior. |
| `trace-missing` | inconclusive | Demonstrates missing trace evidence. |
| `leaky-offline` | fail after redaction | Tests result-row redaction. |

## Built-In Defense Adapters

- `none`: record the model adapter behavior without simulated defense metadata.
- `channel-isolation`: record that source labels and trusted/untrusted channel separation are expected.

## Example Command

```text
python scripts/run_eval_harness.py --model-adapter safe-offline --defense-adapter channel-isolation --case-limit 5
```

## Harness Summary

- Total cases: `5`
- Result rows: `5`
- Tested cases: `5`
- Pass rate: `100.0%`

## Verdict Counts

| Verdict | Count |
|---|---:|
| pass | 5 |
| fail | 0 |
| inconclusive | 0 |
| untested | 0 |
| missing | 0 |

## Failed Cases

No failed cases recorded.

## Missing Results

No missing result rows.

## Surface Summary

| Surface | Pass | Fail | Inconclusive | Untested | Missing |
|---|---:|---:|---:|---:|---:|
| Agent command execution or command-classification boundary | 1 | 0 | 0 | 0 | 0 |
| LLM-enabled agent security benchmark or survey surface | 3 | 0 | 0 | 0 | 0 |
| Prompt-bearing agent instruction boundary | 1 | 0 | 0 | 0 | 0 |