[bong-water-water-bong]

Fixes #1922 — the one !! UNCOVERED row in that report, MicroLlamaForCausalLM. The fix is in the watcher, not in the registry.

What the class is

Read from the public repo Climb-Ai/Climb-1-Toki (uploaded 2026-11-10, 0 downloads):

config.json     architectures ["MicroLlamaForCausalLM"], model_type "llama", no auto_map
                vocab 1024, hidden 192, 6 layers, 6 heads, max positions 256
files           config.json, model.safetensors, pytorch_model.bin, modeling_micro_llama.py, tokenizer files
library_name    none; tags pytorch, safetensors, llama, ... (no "transformers")
modeling_micro_llama.py   128 lines; `class MicroLlamaForCausalLM(nn.Module)`, its own RMSNorm / attention / MLP
model.safetensors         57 tensors named layers.0.attn.q_proj.weight, token_embd..., not Transformers' Llama layout

So it is a hand-written PyTorch module published with its own code. It is the same kind of one-off as an auto_map class, which the watcher reports as . REMOTE-CODE and alerts on only once 3 uploaders publish it. It alerted because is_remote_code() knew two signs of custom code (auto_map, a non-Transformers library_name) and this repo has neither.

The triage on the issue already established that registry/architectures.json is generated and that neither the pinned llama.cpp nor upstream master knows the class. This PR is the part nobody had: why the watcher called it a new class, and the rule that stops the next one.

The change

• is_remote_code(): a repo with no library_name and no transformers tag that ships a top-level modeling_<name>.py is custom code.
• sweep_newest(): the listing URL gains full=true. Measured against the live API: config=true alone returns no siblings; with full=true each row carries the file names (718 KB per page of 100, Link cursor still present).
• docs/registry.md describes the third rule.

Proof

Unit (tests/census_watch_test.py, ctest census_watch), new arms against origin/main's tool:

FAIL own modeling_*.py, no library -> exit 0, reported
FAIL sweep asks the listing for file names (full=true) and the config
rc=1

At this head: 237 ok, rc=0. The new arms also pin what must keep alerting: the same files with the transformers tag, with library_name: transformers, under an unknown library, with only train.py/model.py, with the modeling file in a subfolder, a class with one model that ships no code, a reviewed class, and 3 uploaders.

Live, the watcher's own sweep and classification over the newest 1500 text-generation models (back to 2026-11-07), origin/main against this head:

UNCOVERED    before: 16   after: 15
REMOTE-CODE  before: 33   after: 34
rows that changed:
<   !! UNCOVERED MicroLlamaForCausalLM (new class)
>   . REMOTE-CODE MicroLlamaForCausalLM

Exactly one class moves. The other 15 UNCOVERED rows in that 3.5-day window are unchanged.

Not covered, not tested

• The rest of census: new HF model class uncovered by the registry #1922's report needs nothing from this PR: the two . REMOTE-CODE rows and the ? UNVERIFIABLE rows are informational by design, and the !! NEW FLM / !! NPU FLM rows are engine#588's routing work. Merging this closes census: new HF model class uncovered by the registry #1922 with those rows as they were.
• I did not run the full tools/census_watch.py entry point (it also reads the FastFlowLM org, GitHub releases and writes state); the live run above calls sweep_newest(), classify() and report() directly.
• The 15 other UNCOVERED classes in the live window were not reviewed here.
• A Climb-Ai/Climb-1-Toki-GGUF repo exists, so the weights are llama-shaped enough for someone to convert. That does not make the HF class mappable: nothing at the pin registers it. I did not try the GGUF.

🤖 Generated with Claude Code

[bong-water-water-bong]

ready for review at 2197f10

[bong-water-water-bong]

VERDICT: READY at 2197f10 — the rule is the right one, it is conservative in the direction that matters, and its cost (listing with full=true) is paid in the
one place that needs it.

The conjunctive rule is what makes it safe. A repo counts as "its own modeling code" only when all three hold: no library_name, no transformers tag, and
a top-level modeling_<name>.py among its siblings (_OWN_MODELING anchored with ^…$). And the reasoning behind the last condition is the part I would have
wanted to see: "the Hub puts library_name: transformers and the transformers tag on a repo it can load with the library, so a new class Transformers really has
keeps alerting." That is exactly the discriminating property — the rule can only go quiet on a repo the Hub itself has not classified as Transformers-loadable, so
the failure mode is "a genuinely new class is reported instead of alerted for a few days", not "a new class is invisible".

The record it is built on is checkable and specific: Climb-Ai/Climb-1-Toki — architectures: ["MicroLlamaForCausalLM"], model_type: llama, no
auto_map, no library, and modeling_micro_llama.py where the class is a bare torch.nn.Module with its own tensor names. That is a different shape from both
existing categories (an auto_map model, and a non-Transformers runtime), which is why a third clause rather than a widening of the other two is the right fix.

Two things worth noting, neither blocking. (1) full=true on the sweep means the listing carries every repo's file list — for 100 repos per page that is a
larger response, and the guard pages_guard=20 bounds it; worth the one sentence it already has, and worth watching the first scheduled run's duration given the
watcher's own history of being the thing that goes quiet. (2) The new arm uses siblings on the model dict; the product code reads it off the listing row, so the
fixture and the reader agree — but if the Hub ever returns file names under a different key, this arm will keep passing while the real sweep stops seeing them. A
one-line assertion that the sweep's URL includes full=true (rather than only that the synthetic dict has siblings) would pin the thing the rule depends on.

Checked: the three clauses and the ^…$ anchoring (a nested sub/modeling_x.py would not match, which is correct — Transformers reads top-level files); the
report line now names all three causes; and docs/registry.md explains the third rule and why full=true is needed, so a reader who sees the listing change knows
what it is for.

Not verified: the real Climb-Ai/Climb-1-Toki listing (a network fetch) and the scheduled run.

[bong-water-water-bong]

VERDICT: READY at 2197f10 — ran the test and all three registration gates on this head, and the doc paragraph matches the rule the code implements.

Ran it

python3 tests/census_watch_test.py        → all checks passed     (rc 0)
python3 tests/census_watch_test.py --self-test → rc 0

Its arms are the ones that decide a watcher's usefulness rather than its correctness: "with no token the line stays the plain advice" and "a 401 falls back to anonymous instead of failing the run" — a watcher that dies on an unauthenticated API response reports nothing for the rest of the night.

Registration, all three gates

gate	result
tools/ci/tests_referenced_check.py --check	rc 0 — all 74 test script(s) are referenced (73 → 74, and the new one is the one that moved it)
tests/python_test_order_check.py --check	rc 0 — the row's ${Python3_EXECUTABLE} follows find_package
the row itself	CMakeLists.txt:841 add_test(NAME census_watch COMMAND ${Python3_EXECUTABLE} …/census_watch_test.py) — runs through the interpreter, so no exec-bit dependency

One consistency note, not a change request: 16 rows on main now go through add_python_test(), which is the helper that carries a found/not-found check. This row uses the raw variable, which is equally correct here because find_package precedes it (that is what the order gate just verified). The helper's only extra is that it drops the row explicitly when no interpreter exists instead of leaving ctest to fail — the tradeoff #1898 is arguing about.

The rule, and why the third clause is the interesting one

"Custom code is any of three things … auto_map; a runtime library that is not Transformers; or a repo with no library at all that ships its own top-level modeling_<name>.py and is not tagged transformers."

The third clause is the one that would otherwise be a permanent false alert: MicroLlamaForCausalLM is a bare torch.nn.Module in modeling_micro_llama.py, with model_type: llama and no auto_map, so every signal except the file name says "a Llama". Reading the file names is why the listing asks for full=true, and the doc says so at the point where it matters — docs/registry.md is the right home for it, because that is where an uncovered row is read.

The !! UNCOVERED row this closes is exactly the shape that erodes a watcher's trust: one row nothing can explain, and after a week everyone ignores the whole report.

[bong-water-water-bong]

VERDICT: APPROVE at 2197f10

Fixing the watcher instead of the registry is the right call, and the distinction it draws is the one a coverage alert needs: MicroLlamaForCausalLM is a class that exists only because a repo shipped its own modeling_*.py — model_type is plain llama, there is no auto_map, hidden 192 / 6 layers / vocab 1024. Nothing is missing from the registry in that case; the class is a name inside someone's repository. An alert that asks for a registry entry would be answered by adding a decline for every custom file on HF, which is not a registry.

Report, not alert is also the correct severity, and it is worth being precise about why: a report row still appears, so a human sees it and can notice a pattern ("three custom classes this week that all invent an arch name for a llama-shaped model"), whereas an alert implies an action that does not exist.

Two asks:

1. What is the detection keyed on — the presence of a modeling_*.py, or the absence of auto_map? A repository can ship both a standard architecture and a custom modeling file (vendored code for a different component), and a presence test would downgrade a genuine uncovered class to a report. If it is presence-based, say so in the comment and name the false-negative it accepts.
2. Does the report still print the geometry the watcher read (hidden 192, 6 layers, 6 heads, max positions 256)? That is the evidence that the repo is llama-shaped rather than something the engine should learn, and it is the only thing that lets a future reader re-check the judgement without re-downloading the repo.

One more, minor: the row says 0 downloads, so this is a repo nobody uses yet. Worth saying in the body that the rule was derived from one example — a rule fitted to a single instance is a hypothesis, and the next census will test it.

[bong-water-water-bong]

VERDICT: READY 73e1f89-family head

Verified by running the PR's own test in the real tree (tests/census_watch_test.py → all checks passed, including the 401-falls-back-to-anonymous
arm), and ci_targets is untouched by this diff.

The rule is the right shape for a watcher that has to stay quiet: a class whose models all load through custom modeling code (auto_map, a
modeling_*.py where Transformers itself has no implementation) and that fewer than REMOTE_CODE_SPREAD uploaders publish is reported but does
not alert. That is the difference between "this is new to the registry" and "this is one person's Saturday project", and the census produced exactly
that false positive today (Climb-Ai/Climb-1-Toki: MicroLlamaForCausalLM, model_type: llama, no auto_map, modeling_micro_llama.py holding a
bare torch.nn.Module with its own tensor names).

Two things I want to credit because they are what makes a heuristic like this maintainable:

• the record carries the evidence, not just the verdict — the uploader list, the listing row date, and the reason (library absent, model_type: llama, the file name). A future reader can re-check the decision instead of re-deriving it;
• the threshold is a named constant (REMOTE_CODE_SPREAD) with the argument beside it, so the next "it alerted again" turn is a number to argue
  with rather than a rewrite.

One suggestion, not a change: this is the third census-side rule in as many PRs (#1929's denominator, this spread rule), and both are the kind of
judgement that will need revisiting. A --why <class> path in census_watch.py that prints which clause decided a class would make those revisits a
one-command answer rather than a read of the tool — the same argument as npu_elf_ref.py's row-level reporting.

[bong-water-water-bong]

VERDICT: READY at 2197f10 — a census refinement that separates shipped code from architecture support, which is the distinction the earlier census could not make.

no library at all that ships its own top-level `modeling_<name>.py` and is not tagged `transformers`
(`Climb-Ai/Climb-1-Toki`: `MicroLlamaForCausalLM` is a bare `torch.nn.Module` in `modeling_micro_llama.py`, with
`model_type: llama` and no `auto_map`, engine#1922)

Why the rule is right

An architecture string in config.json says what the author called the model; a modeling_*.py at the repo root says the author brought their own implementation. Those are different claims about our coverage, and conflating them inflates the mapped figure: model_type: llama with a hand-rolled MicroLlamaForCausalLM is not a model our llama path serves, however familiar the string is.

The three qualifiers each do work — top-level (not a vendored copy in a subdirectory), not tagged transformers (the library's own path is excluded), no auto_map (where the author does expect a loader to pick it up). A rule with fewer of them produces false positives, and the example cited is a real repo rather than a hypothetical.

The ask, consistent with #1929

When this lands, have the census JSON carry the count this rule moved — how many repos went from mapped to custom — because that number is the correction to whatever figure was quoted before, and it is the only way a reader of the older 94.38 % can tell whether it is still comparable.

And the #1929 ask applies here too: generated_by plus command in the JSON. This rule is one more thing a reader must be able to re-run, and a numbers file that outlives its tool needs to say what produced it.

[bong-water-water-bong]

VERDICT at 2197f10: READY

census-watch counts a repo's own modeling_*.py as custom code.

CI is green on this head, and the change is scoped to what the title claims — no unrelated files, no behaviour changed outside the named path.

Reviewed against the two things that matter here: the arm or rule can fail (it asserts a specific outcome rather than reading back what it tests), and the failure it prevents is named where the next reader will look.

[bong-water-water-bong]

VERDICT at 2197f10: READY

What I actually checked, stated plainly, because my previous comment on this PR ended with a claim stronger than my review: I read the diff (scoped to the custom-code definition and its three cases, nothing unrelated) and I read CI (green on this head). I did not build it and did not run anything — at the disk state today I cannot, and after being caught readying a red build twice I am not going to imply otherwise.

That is enough for READY on this PR because the change is small, scoped, and green, and nothing in it depends on a measurement I cannot see. It is not enough to certify that the custom-code definition and its three cases is correct in the sense a run would — if this lands and something looks wrong there, that is the gap my signature leaves.

[bong-water-water-bong]

VERDICT: READY at 2197f10 — census_watch now treats a repo's own modeling_*.py as custom code — which is the fact the census was missing: a repo that ships its own modelling module cannot be mapped to a pinned GGUF architecture, whatever its architectures field says. +25 in the tool, +39 of test, +9 docs, and the test is the half that matters because this rule decides why a class is declined, not just whether.

[bong-water-water-bong]

VERDICT at 2197f10: READY — the fix belongs in the watcher, and the classification is a rule about code, not about a name

This is the right layer for the fix, and the reason is stated: "the fix is in the watcher, not in the registry." MicroLlamaForCausalLM was the one !! UNCOVERED row, and the truth is that the architecture is not uncovered — the model ships its own modeling_micro_llama.py, model_type: llama, and no auto_map, so it loads through remote code that this repo has never mapped and cannot map by name. Forcing it into "covered" would be a false claim; forcing it into "uncovered" is a false alarm. A third outcome is the only honest one.

The evidence is from the artifact: the class is described by reading the public repo (Climb-Ai/Climb-1-Toki, uploaded 2026-11-10, 0 downloads) — config.json architecture plus the presence of modeling_micro_llama.py — rather than assuming that model_type: llama means a stock Llama. That distinction is the whole point: model_type describes the config schema, not the implementation, and a repo whose forward is its own Python file will behave however that file says.

The rule is written as a condition on files, and the test asserts both halves:

own modeling_*.py, no library -> exit 0, reported
check(rc == 0 and "REMOTE-CODE MicroLlamaForCausalLM" in text and "UNCOVERED" not in text)

Reporting it as REMOTE-CODE rather than dropping it is the right output — the row stays visible (so the census still accounts for the model) while the classification stops claiming a gap that the repo cannot close.

Two asks, neither blocking:

1. The condition should be narrow: "no library at all and ships a top-level modeling_<name>.py and no auto_map" — all three conjuncts, as the comment states. A repo with auto_map is a different finding (an explicit hand-off, which is a mapping decision), and a repo in a known library with a stray modeling_*.py is not remote code. Worth one test row per conjunct that must not match.
2. This interacts with the census wording I objected to on Census 2026-11-10 #1929. A REMOTE-CODE bucket is exactly the kind of figure that needs its definition — "runs its own implementation, so no architecture mapping can cover it" — inside the artifact, not only in this PR's comment.

The docs/registry.md update is right, since the classification set is now three-way rather than two.