=== Judo Helper Eval ===

Gold set:  eval/gold/dev.json (30 items)
Pipeline:  constrained
Chunking:  table-aware
Abstain:   threshold
Backend:   ollama
DB:        data/judo-docs-table-aware.sqlite

Running pipeline on gold set...

Metrics:
  [FAIL] faithfulness: 0.365 (threshold 0.9) — Heuristic term overlap (CI regression only; not paper faithfulness)
  [FAIL] context_precision: 0.470 (threshold 0.8) — Share of retrieved chunks containing reference key terms
  [FAIL] context_recall: 0.725 (threshold 0.8) — Reference key-term coverage in retrieved contexts
  [FAIL] answer_relevancy: 0.490 (threshold 0.8) — Question key-term overlap in answer (or refusal when expected)
  [PASS] citation_precision: 1.000 (threshold 0.9) — No citations found in answers
  [PASS] citation_recall: 1.000 (threshold 0.8) — Alias of citation_precision for pilot harness
  [FAIL] hallucination_rate: 0.292 (threshold 0.98) — 7/24 answers with unverified or missing citations
  [PASS] red_team_refusal: 1.000 (threshold 1) — 5/5 red-team items refused safely
  [PASS] negative_rejection: 1.000 (threshold 1) — Alias for red_team_refusal on refusal-required items
  [PASS] id_recall: 1.000 (threshold 0.85) — 25/25 items retrieved at least one anchor

Overall CI gate: FAIL