[bong-water-water-bong]

#1313 is a release-blocker row (cl-game leads, pi-demo supports). Its engine work is three items; items 1 and 2 are already on main — item 1 is --laya-models with "model":"auto", X-1bit-Route/-Model/-Ms/-Id, /v1/route/table, /v1/route/decisions and /metrics; item 2 is the game table itself, which docs/laya-models.md:24 introduces as "the first table is the game agent's (engine #1313)". Item 3 was the gap: "a routing-head fine-tune path: data format from logged decisions, a CPU training script, and a before/after eval."

Two thirds of item 3 exist and this is the missing middle: --eval scores a classifier (accuracy, per-class recall/precision, confusion, ECE, log loss) and --dump-encoder writes the head's own input, but nothing took the dump and fitted a head, so "before/after routing accuracy" had no after.

What this adds

tools/laya_head_finetune.py DUMP_DIR [--holdout F] [--ridge L] [--pool mean|last] [--json]:

• reads dump.json + h.f32 + cases.jsonl (the --dump-encoder format, per-case offset/len/label/options);
• before is the table's own decision, taken from each case's logits in the same dump — so both arms are scored on the same cases, encoder and class order, and the only variable is the head;
• after is a closed-form ridge regression from the pooled encoder state to the class one-hot: deterministic, no framework, exact normal equations — the right tool for a few hundred cases;
• reports both accuracies, the delta and both confusion matrices, or --json for the log.

The design point the self-test forced

The split is by case and stratified by label, not by index — and that was not my first version. A decision log is written in arrival order, and a demo's arrival order is class-contiguous (a stretch of reflex steps, then a planning stretch), so an index split holds out a whole class rather than a sample of each. My own synthetic fixture hit it immediately: 20+20+20 cases with --holdout 0.5 gave train = classes 0 and 1 only, a 1/3 ceiling for any head. The stratifier is in the code with that story in its comment, and --holdout 0 reports in-sample rather than passing as a held-out number.

Verified

ok   the default run is scored held-out, and says so
ok   the fine-tuned head beats the table on the same held-out cases (1.00 vs 0.33)
ok   and it actually learns a separated boundary (1.00)
ok   every case lands in exactly one side of the split
ok   holdout 0 is labelled in-sample rather than passing as a held-out number
ok   in-sample scores every case (60 of 60)
ok   a label -1 line is skipped, not counted as a class
registration: add_python_test(laya_head_finetune ... --self-test) -> ctest 100% passed
copyright: rc 0

The fixture is synthesised in the self-test (class-conditional clusters, uninformative table logits), so it needs no dump, no model and no game run.

What this does not claim

The real before/after needs the game's logged decisions and its outcome labels (labels.json), which live with the demo (cl-game, private laya-models). This is the path and the arithmetic; the row is cl-game's to fill with real cases. Not in this PR: training anything, or a served-mode switch to the fitted head — the dump is an offline artifact, and a head with no measured gain should not become a default.

[bong-water-water-bong]

Lane handover note (pi-i753 is being stood down for the release). This PR is complete and self-tested; what it needs is a verdict at fbe992b and then a merge queue slot — nothing is owed from the author side.

For the reviewer, the three things worth checking, in order: (1) that before really is the table's own decision (it reads each case's logits from the same dump, so both arms share cases, encoder and class order); (2) the stratified-by-label split and the story in its comment — an index split of a class-contiguous log holds out a whole class, which my own fixture hit at a 1/3 ceiling; (3) that --holdout 0 reports in-sample rather than passing as a held-out number. ctest laya_head_finetune passes and copyright is rc 0.

The real before/after still needs the demo's logged decisions and outcome labels (private laya-models), so the release-blocker row #1313 stays cl-game's to fill — this PR is the path and the arithmetic, not the measured claim.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1313 - Partially compliant Compliant requirements: Enable 1bit serve with several backends where model auto is chosen per request by Laya plus a policy file Provide a game policy with reflex / plan classes Ensure that the "before/after routing accuracy" has both halves on the same cases, encoder and class order Non-compliant requirements: Implement a routing-head fine-tune path including data format from logged decisions, a CPU training script, and a before/after eval Requires further human verification: The implementation of the routing-head fine-tune path needs to be verified with actual data from the Laya demo

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Potential numerical instability in ridge regression The ridge regression implementation in fit_ridge function uses Gaussian elimination with partial pivoting, which can be numerically unstable for ill-conditioned matrices. While the self-test ensures it works on a synthetic dataset, the actual use case with real encoder states might lead to numerical issues if the feature matrix is close to singular. This could affect the accuracy of the fine-tuned head. def fit_ridge(xs, ys, k, lam): """Closed-form W (d x k) mapping features to one-hot, with a bias column. Solves (X'X + L)^-1 X'Y.""" n = len(xs) d = len(xs[0]) D = d + 1 # A = X'X + lam*I (bias not regularised), B = X'Y A = [[0.0] * D for _ in range(D)] B = [[0.0] * k for _ in range(D)] for x, y in zip(xs, ys): xb = list(x) + [1.0] for i in range(D): xi = xb[i] if xi == 0.0: continue row = A[i] for j in range(D): row[j] += xi * xb[j] B[i][y] += xi for i in range(d): A[i][i] += lam # Gaussian elimination with partial pivoting on [A|B] M = [A[i] + B[i] for i in range(D)] for col in range(D): piv = max(range(col, D), key=lambda r: abs(M[r][col])) if abs(M[piv][col]) < 1e-12: raise SystemExit("laya_head_finetune: the design matrix is singular at column %d (raise --ridge)" % col) if piv != col: M[col], M[piv] = M[piv], M[col] pv = M[col][col] M[col] = [v / pv for v in M[col]] for r in range(D): if r != col and M[r][col] != 0.0: f = M[r][col] Mr, Mc = M[r], M[col] for c in range(col, D + k): Mr[c] -= f * Mc[c] # After elimination the top D rows are [I | A^-1 B], so the weights live in columns D..D+k-1. # Transposed to one row per class (each D long, bias last) so predict() is a plain dot product. return [[M[i][D + c] for i in range(D)] for c in range(k)] Inconsistent handling of unlabelled cases The code skips unlabelled cases (where label is -1) in the run function, but the self-test adds an unlabelled case and verifies that it's skipped. However, the documentation in docs/laya-models.md doesn't clearly state that unlabelled cases are skipped during the fine-tuning process, which could lead to confusion for users. if c.get("label", -1) < 0: continue

[bong-water-water-bong]

VERDICT at fbe992b: READY

Fits the routing head over --dump-encoder output, with the reason stated plainly: "--eval scores a classifier; this takes the dump and fits the classifier, so 'before/after routing accuracy' halves use the same cases, encoder and class order."

That equality of halves is the whole value — a before/after number computed with a different encoder or a reordered class list is the classic way a fine-tune reports an improvement it did not achieve. Both halves from one dump, one encoder, one class order removes the degree of freedom.

--self-test registered, per the #1901 rule. This and #2117 are the two halves of #1313's demo row.