
==============================================================================
FAITHFULNESS GAP  C - A   (items answered by BOTH, defined faith in BOTH)
==============================================================================

-- Era J2  (judge qwen2.5:14b) --
rep     n     C mean     A mean      gap     95% pctile interval           W/L/T
r1     92     0.9725     0.9203  +0.0522   [+0.0123, +0.0993]        6/1/85
r2     97     0.9670     0.8935  +0.0735   [+0.0220, +0.1282]       10/3/84
r3     91     0.9750     0.9493  +0.0257   [-0.0075, +0.0660]        6/3/82
mean                             +0.0505   (SD across replicates 0.0240)
pooled question-mean cluster bootstrap: n_q=102  gap=+0.0614  [+0.0234, +0.1074]
direction: C more faithful

-- Era J3  (judge mistral-nemo:12b) --
rep     n     C mean     A mean      gap     95% pctile interval           W/L/T
r1     94     0.9894     0.9468  +0.0426   [+0.0000, +0.0957]        5/1/88
r2     98     0.9898     0.9388  +0.0510   [+0.0000, +0.1020]        6/1/91
r3     97     0.9794     0.9273  +0.0521   [-0.0087, +0.1139]        7/2/88
mean                             +0.0486   (SD across replicates 0.0052)
pooled question-mean cluster bootstrap: n_q=106  gap=+0.0615  [+0.0145, +0.1118]
direction: C more faithful

==============================================================================
PER-ITEM J2 vs J3 AGREEMENT (answered factual items, both judges scored)
==============================================================================
arm rep      n  mean J2  mean J3     exact  within .2  binary agr     rho
C   r1     103   0.9697   0.9806    96/103     99/103      96/103   0.255
C   r2     106   0.9698   0.9811   100/106    101/106     100/106   0.277
C   r3     102   0.9692   0.9804    95/102     97/102      95/102   0.258
A   r1      97   0.9381   0.9691    91/97      91/97       91/97    0.420
A   r2     102   0.8987   0.9461    94/102     94/102      94/102   0.580
A   r3     100   0.9588   0.9361    91/100     92/100      93/100   0.503
