OK   gold full.json sha256 d55250b24948… == manifests/gold-full.sha256 d55250b24948…
OK   gold has 150 items (expect 150)
OK   rq1-C-r1 resolves -> 2026-08-16T13-06-02-903Z_69f4783d-2a08-4b9f-854d-72c41d511a2d
OK   rq1-C-r1 report has 150 pipeline_results
OK   rq1-C-r1 item_count == 150
OK   rq1-C-r1 result ids == gold ids (missing 0, extra 0)
OK   rq1-C-r1 report.prerefuse == 'none'
OK   rq1-C-r1 report.seed == 42 (expect 42)
OK   rq1-C-r1 report.pipeline == 'constrained' (expect 'constrained')
OK   rq1-C-r1 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-C-r1 manifest env OLLAMA_SEED == 42
OK   rq1-C-r1 judged convention == 'claims-v2.1'
OK   rq1-C-r1 judged faithfulness_verdicts is dict of 150
OK   rq1-C-r1 judged relevancy_judge is dict of 150
OK   rq1-C-r1 judged source_run_id == report run_id (69f4783d)
OK   rq1-C-r1 latency.csv has 150 data rows
OK   rq1-C-r2 resolves -> 2026-08-16T14-06-38-608Z_8eeeadab-66f3-4424-83ef-e63976aa49a5
OK   rq1-C-r2 report has 150 pipeline_results
OK   rq1-C-r2 item_count == 150
OK   rq1-C-r2 result ids == gold ids (missing 0, extra 0)
OK   rq1-C-r2 report.prerefuse == 'none'
OK   rq1-C-r2 report.seed == 43 (expect 43)
OK   rq1-C-r2 report.pipeline == 'constrained' (expect 'constrained')
OK   rq1-C-r2 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-C-r2 manifest env OLLAMA_SEED == 43
OK   rq1-C-r2 judged convention == 'claims-v2.1'
OK   rq1-C-r2 judged faithfulness_verdicts is dict of 150
OK   rq1-C-r2 judged relevancy_judge is dict of 150
OK   rq1-C-r2 judged source_run_id == report run_id (8eeeadab)
OK   rq1-C-r2 latency.csv has 150 data rows
OK   rq1-C-r3 resolves -> 2026-08-16T15-16-51-566Z_5da8a8c3-c4ef-4557-83b7-96a2189c71ae
OK   rq1-C-r3 report has 150 pipeline_results
OK   rq1-C-r3 item_count == 150
OK   rq1-C-r3 result ids == gold ids (missing 0, extra 0)
OK   rq1-C-r3 report.prerefuse == 'none'
OK   rq1-C-r3 report.seed == 44 (expect 44)
OK   rq1-C-r3 report.pipeline == 'constrained' (expect 'constrained')
OK   rq1-C-r3 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-C-r3 manifest env OLLAMA_SEED == 44
OK   rq1-C-r3 judged convention == 'claims-v2.1'
OK   rq1-C-r3 judged faithfulness_verdicts is dict of 150
OK   rq1-C-r3 judged relevancy_judge is dict of 150
OK   rq1-C-r3 judged source_run_id == report run_id (5da8a8c3)
OK   rq1-C-r3 latency.csv has 150 data rows
OK   rq1-A-r1 resolves -> 2026-08-16T13-18-42-757Z_c9c219ab-cfca-4ed3-8c34-c56c63a420bf
OK   rq1-A-r1 report has 150 pipeline_results
OK   rq1-A-r1 item_count == 150
OK   rq1-A-r1 result ids == gold ids (missing 0, extra 0)
OK   rq1-A-r1 report.prerefuse == 'none'
OK   rq1-A-r1 report.seed == 42 (expect 42)
OK   rq1-A-r1 report.pipeline == 'agentic-react' (expect 'agentic-react')
OK   rq1-A-r1 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-A-r1 manifest env OLLAMA_SEED == 42
OK   rq1-A-r1 judged convention == 'claims-v2.1'
OK   rq1-A-r1 judged faithfulness_verdicts is dict of 150
OK   rq1-A-r1 judged relevancy_judge is dict of 150
OK   rq1-A-r1 judged source_run_id == report run_id (c9c219ab)
OK   rq1-A-r1 latency.csv has 150 data rows
OK   rq1-A-r2 resolves -> 2026-08-16T14-20-04-054Z_3924cf53-4df1-4f48-b4dc-d2c77f59b84c
OK   rq1-A-r2 report has 150 pipeline_results
OK   rq1-A-r2 item_count == 150
OK   rq1-A-r2 result ids == gold ids (missing 0, extra 0)
OK   rq1-A-r2 report.prerefuse == 'none'
OK   rq1-A-r2 report.seed == 43 (expect 43)
OK   rq1-A-r2 report.pipeline == 'agentic-react' (expect 'agentic-react')
OK   rq1-A-r2 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-A-r2 manifest env OLLAMA_SEED == 43
OK   rq1-A-r2 judged convention == 'claims-v2.1'
OK   rq1-A-r2 judged faithfulness_verdicts is dict of 150
OK   rq1-A-r2 judged relevancy_judge is dict of 150
OK   rq1-A-r2 judged source_run_id == report run_id (3924cf53)
OK   rq1-A-r2 latency.csv has 150 data rows
OK   rq1-A-r3 resolves -> 2026-08-16T15-30-07-037Z_22048d1a-8c80-4fc9-928b-6afe442ff41c
OK   rq1-A-r3 report has 150 pipeline_results
OK   rq1-A-r3 item_count == 150
OK   rq1-A-r3 result ids == gold ids (missing 0, extra 0)
OK   rq1-A-r3 report.prerefuse == 'none'
OK   rq1-A-r3 report.seed == 44 (expect 44)
OK   rq1-A-r3 report.pipeline == 'agentic-react' (expect 'agentic-react')
OK   rq1-A-r3 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-A-r3 manifest env OLLAMA_SEED == 44
OK   rq1-A-r3 judged convention == 'claims-v2.1'
OK   rq1-A-r3 judged faithfulness_verdicts is dict of 150
OK   rq1-A-r3 judged relevancy_judge is dict of 150
OK   rq1-A-r3 judged source_run_id == report run_id (22048d1a)
OK   rq1-A-r3 latency.csv has 150 data rows
OK   rq1-T-r1 resolves -> 2026-08-16T13-27-10-095Z_73f9fb02-2ca0-4945-a334-9e040bcd1d50
OK   rq1-T-r1 report has 150 pipeline_results
OK   rq1-T-r1 item_count == 150
OK   rq1-T-r1 result ids == gold ids (missing 0, extra 0)
OK   rq1-T-r1 report.prerefuse == 'none'
OK   rq1-T-r1 report.seed == 42 (expect 42)
OK   rq1-T-r1 report.pipeline == 'two-tier' (expect 'two-tier')
OK   rq1-T-r1 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-T-r1 manifest env OLLAMA_SEED == 42
OK   rq1-T-r1 judged convention == 'claims-v2.1'
OK   rq1-T-r1 judged faithfulness_verdicts is dict of 150
OK   rq1-T-r1 judged relevancy_judge is dict of 150
OK   rq1-T-r1 judged source_run_id == report run_id (73f9fb02)
OK   rq1-T-r1 latency.csv has 150 data rows
OK   rq1-T-r2 resolves -> 2026-08-16T14-29-20-384Z_39aadf6d-1d14-46a8-8316-b1b937fadc52
OK   rq1-T-r2 report has 150 pipeline_results
OK   rq1-T-r2 item_count == 150
OK   rq1-T-r2 result ids == gold ids (missing 0, extra 0)
OK   rq1-T-r2 report.prerefuse == 'none'
OK   rq1-T-r2 report.seed == 43 (expect 43)
OK   rq1-T-r2 report.pipeline == 'two-tier' (expect 'two-tier')
OK   rq1-T-r2 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-T-r2 manifest env OLLAMA_SEED == 43
OK   rq1-T-r2 judged convention == 'claims-v2.1'
OK   rq1-T-r2 judged faithfulness_verdicts is dict of 150
OK   rq1-T-r2 judged relevancy_judge is dict of 150
OK   rq1-T-r2 judged source_run_id == report run_id (39aadf6d)
OK   rq1-T-r2 latency.csv has 150 data rows
OK   rq1-T-r3 resolves -> 2026-08-16T15-38-39-479Z_1c4fd35a-d9d4-4384-a358-ce4254450d94
OK   rq1-T-r3 report has 150 pipeline_results
OK   rq1-T-r3 item_count == 150
OK   rq1-T-r3 result ids == gold ids (missing 0, extra 0)
OK   rq1-T-r3 report.prerefuse == 'none'
OK   rq1-T-r3 report.seed == 44 (expect 44)
OK   rq1-T-r3 report.pipeline == 'two-tier' (expect 'two-tier')
OK   rq1-T-r3 manifest.git_commit == eaffacd7292c… (freeze eaffacd7292c…)
OK   rq1-T-r3 manifest env OLLAMA_SEED == 44
OK   rq1-T-r3 judged convention == 'claims-v2.1'
OK   rq1-T-r3 judged faithfulness_verdicts is dict of 150
OK   rq1-T-r3 judged relevancy_judge is dict of 150
OK   rq1-T-r3 judged source_run_id == report run_id (1c4fd35a)
OK   rq1-T-r3 latency.csv has 150 data rows
OK   single judge convention across all 9: ['claims-v2.1']
OK   single judge model across all 9: ['qwen2.5:14b']

ALL INPUT CHECKS PASSED
