[caiovicentino]

Second follow-up to #62 (Ekbasis-Skopos-27B), after #63.

Jev against Ekbasis-27B and Ekbasis-Skopos-27B (not pre-registered, measured 2026-10-16; the owner decided to publish
it). TypeSafe AI's general-purpose evaluator (typesafe-ai/jev), zero-shot, in two wordings of each question, on the
release protocol and the five suites of the hosted scorecard, next to the per-item predictions both models published. On
git v3 (all 3,105 items) Jev's headline wording answers 90.0% right, against 96.8% for Skopos INT4 and 96.4% for
Ekbasis-27B INT4. On the guard's larger set, known command types, it raises 733 false alarms among 863 safe commands at
0.2, against 25 for Skopos INT4 (AUROC 0.879 against 0.992). Its strength: very few confident errors (0.00 to 0.75 wrong
at confidence ≥ 0.9 per 100 on the scorecard suites, against 1.25 to 14.50 for the Ekbasis builds), from probabilities
compressed toward the middle; and it costs less per check (git v3 known: US$21.93 per million checks measured, against
US$89.66 estimated for the hosted Ekbasis). results_skopos/jev_comparison/ is the package as written, with every
per-item answer; its scripts/recompute.py reproduces results.json. A section of RELEASE_EVAL_SKOPOS.md; a
CHANGELOG entry.

Addendum A7 (prereg_skopos/PREREG_ADDENDUM.md, registered before any A7 request): confident errors with every
option's probability on the confident-errors items, Skopos INT4 against Ekbasis-27B INT4, raw and at each build's
calibrated temperature (1.67 and 1.55). The per-item files, the analysis and the script that wrote them are in
results_skopos/a7/; a section of RELEASE_EVAL_SKOPOS.md. The prediction, piece by piece on its paired interval (all
items):

• calibrated lowers confident errors in U, Skopos: held (-3.13 per 100, 95% CI [-3.60, -2.65])
• calibrated lowers confident errors in U, Ekbasis-27B: held (-2.55 per 100, 95% CI [-2.96, -2.13])
• Skopos calibrated at or below Ekbasis-27B raw in U: held (-2.03 per 100, 95% CI [-2.48, -1.58])
• fewer errors in T (Skopos INT4 vs Ekbasis-27B INT4): held (+0.82 accuracy points, 95% CI [+0.39, +1.25])
• fewer errors in H (Skopos INT4 vs Ekbasis-27B INT4): held (+3.36 accuracy points, 95% CI [+2.27, +4.53])
• more confident errors in U (raw): held (+1.10 per 100, 95% CI [+0.65, +1.57])

The prediction compares Skopos calibrated with Ekbasis-27B raw; with both calibrated, in the never-trained families
Skopos INT4 is wrong at confidence ≥ 0.9 on 1.62 per 100 answers against 1.10 (+0.52, 95% CI [+0.26, +0.78]); its
calibrated ECE there is level (0.021 against 0.022).

Not run. A7 on Skopos bf16 was not run. A bf16 run during the phase A battery was stopped because it shared the GPU
(124 items in 12 min; DEVIATIONS_RUN.md). After the battery, the two INT4 builds ran first (they back the hosted API's
scorecard and the switch made on 2026-10-16), and the evaluation machine was then retired by the owner's decision before
a bf16 run. The pre-registered bf16 = A2 consistency check therefore did not run; for information, each INT4 build gives
its own bf16's A2 answer on 97.6% (Skopos) and 97.2% (Ekbasis-27B) of items.

Checks. results_skopos/scripts/verify_summaries.py: 24 of 24 summaries recomputed. RELEASE_EVAL_SKOPOS.md
rebuilt with --final, no change and nothing pending; the documents as scripts/apply_skopos_docs.py writes them; the
pre-registration hashes; the secrets scan of every file the branch adds or changes.

🤖 Generated with Claude Code