Skip to content

docs: the T4 has never run a kernel here - #92

Merged
vyncint merged 1 commit into
mainfrom
docs/limitations-t4-claim
Sep 22, 2026
Merged

vyncint merged 1 commit into
mainfrom
docs/limitations-t4-claim

Conversation

@vyncint

@vyncint vyncint commented Sep 22, 2026

Copy link
Copy Markdown
Owner

docs/LIMITATIONS.md says both of these, twenty lines apart:

line 147: Only 8.6 (A10G) and 7.5 (T4) have ever had a kernel measured on them here.

line 167: All published numbers in this repository are from the A10G; nothing has been measured on a T4.

They cannot both be true, and the evidence in the repository is with the second:

  • docs/research-baseline.md:16 — the tier-2 box is an AWS g5.xlarge, NVIDIA A10G (sm_86, cc 8.6), "chosen over the T4 by the operator; barely above T4 spot"
  • model-calibration.toml:11 — device = "NVIDIA A10G (cc 8.6)", and it is the only device in the file
  • runs/ holds one committed sweep, reduce-stable-metal, on an Apple M4 Pro

So line 147 claimed a hardware validation at cc 7.5 that never happened.

This is not a typo in a footnote. README.md points at this file twice — "the honest edges live in docs/LIMITATIONS.md. Read that before trusting a result" — and the paragraph exists specifically to tell a reader which rows of the model's device table are experience and which are only documented capacity. Someone deciding whether to act on a --cc 7.5 ranking was reading this exact sentence to make that decision, and it told them the part had been measured.

Five of the six rows are capacity rather than experience, not four. The corrected paragraph says so, names the two files that establish it, and keeps a parenthetical recording the contradiction — a claim that was wrong once is worth marking, so nobody restores it from memory.

I checked the rest of the file for the same shape: no other capability claim in it is contradicted by research-baseline.md or model-calibration.toml.

Closes #73

docs/LIMITATIONS.md said both "Only 8.6 (A10G) and 7.5 (T4) have ever had
a kernel measured on them here" and, twenty lines later, "nothing has
been measured on a T4". The evidence is with the second:
docs/research-baseline.md records the tier-2 box as a g5.xlarge with an
A10G, chosen over the T4 by the operator, and model-calibration.toml
names one device.

Not a footnote. README points at this file twice -- read it before
trusting a result -- and the paragraph exists to tell a reader which rows
of the model's device table are experience and which are documented
capacity. Someone deciding whether to act on a --cc 7.5 ranking was
reading a sentence saying it had been validated on silicon. Five of the
six rows are capacity, not four.

Signed-off-by: Vyncint Ng <chivy.nguyen@manabie.com>
@vyncint
vyncint force-pushed the docs/limitations-t4-claim branch from bb0e29f to bf077e9 Compare September 22, 2026 06:14
@vyncint
vyncint merged commit 24ebcc5 into main Sep 22, 2026
11 checks passed
@vyncint
vyncint deleted the docs/limitations-t4-claim branch September 22, 2026 06:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs/LIMITATIONS.md contradicts itself about the T4, and claims a measurement that never happened

2 participants