Skip to content

A generation parameter written on a step is refused, not dropped - #14

Merged
inhuman merged 3 commits into
mainfrom
step-sampling-fields
Aug 15, 2026
Merged

A generation parameter written on a step is refused, not dropped#14
inhuman merged 3 commits into
mainfrom
step-sampling-fields

Conversation

@inhuman

@inhuman inhuman commented Aug 15, 2026

Copy link
Copy Markdown
Owner

max_tokens lives in Sampling. Written at STEP level it was not read — YAML
drops an unknown key without a word:

- name: write_fix
  instruction: 
  max_tokens: 8000      # the engine never saw this

A ceiling declared and absent at the same time, and by eye that is
indistinguishable from a step that never asked for one.

The measurement

Over a live catalogue of 29 skills there were ten declarations of a token
ceiling. Nine were written on the step and had never worked a single day.
The one that worked sat inside a profile's sampling.

Not one author's inattention: those files were written by hand by someone who
knew the format, and a skill-writing model produces the same shape. Every other
parameter of a step — model, tools, max_calls, response_schema,
one_of — is written on the step, so a sampling knob reads as their neighbour.

What this does

Refuses it, with the block and the author's own values printed:

steps[0] (write_fix): max_tokens is a generation parameter and belongs
inside `sampling: {max_tokens: 8000}`, not on the step itself

Refused rather than folded into sampling as an alias: the engine keeps
generation parameters in a block of their own on purpose, and an alias blurs the
line that block exists to draw. A refusal is loud and paid once — a description
is fixed for good, where the silence lasted nine declarations in a row.

The values in the suggestion are the author's, deliberately. The previous
diagnosis of this — from a host repairing descriptions against the schema — said
"an unknown key was removed", about a key that is perfectly known, just one
level down. An author told their key is unknown goes looking for a typo, not for
a block.

How it is caught

An inline *Sampling on Run and on Profile, tagged schema:"-": the parser
reads these keys in order for Validate to refuse them, and the tag says so to
the test that keeps the schema equal to the structs — describing them in the
schema would state the opposite of what Validate does.

Both the check and its test enumerate the fields by reflection over the type.
That is the point rather than a convenience: the defect is that a parameter the
engine does not read disappears in silence, and a hand-written list of seven
names would recreate it for the eighth. Verified by temporarily adding a field —
the case and the refusal both appeared with no edit.

Two things beyond the letter of the task

  • Profiles get the same check. No live case was found there, but a profile
    is where the ceiling that DID work was written, so it is the shape an author
    copies from — and copying it a level too high is exactly this mistake.
  • The check runs before the "step does nothing" verdict, so a step carrying
    nothing but a stray max_tokens gets the diagnosis it needs instead of a
    formally correct and useless one.

Version

2.4.1 — the format gained no field, so by this repository's convention it is
a patch, with the precedent of 2.2.4 ("Fixed, and a behaviour change").

⚠️ A skill carrying one of these now fails to load, with the fix printed.
The ceiling was not being applied before either, so nothing it did changes —
only the lie about it does.

The schema is a second copy of what the format is, and nothing kept it equal to the first: max_tool_errors is read on every step and was described nowhere. A host that repairs a description against the schema then drops a legal field, quietly.

The test compares yaml tags with the schema's properties, so the next field added to one of them fails here rather than in somebody's skill.
kubectl_get answers {"items": [...]}, the skill said for_each in: pods, and the value - multi-line and not starting with [ - was split BY LINES. The first item was the line {, and the failure surfaced a step deeper, on a field access, where nothing points back at the loop.

An object with one list is now iterated as that list and the step is marked degraded: guessing for the author has to be audible. An object with several is refused, naming the fields and the author's own variable in the example.
max_tokens на уровне шага YAML выбрасывал молча: потолок объявлен и
одновременно отсутствует, а на глаз это неотличимо от шага, который потолка
не просил. Замер по живому каталогу из 29 скиллов: из десяти объявлений
потолка девять написаны на шаге и не действовали ни дня; единственное
работающее лежало в sampling профиля.

Это не невнимательность одного автора. Те файлы писал руками человек,
знающий формат, и сборщик скиллов генерирует ту же форму: остальные
параметры шага (model, tools, max_calls, response_schema, one_of) пишутся
именно на шаге, и ручка сэмплирования читается как их сосед.

Из двух вариантов выбран отказ, а не слияние в sampling: движок держит
параметры генерации отдельным блоком намеренно, и алиас размывает границу,
ради которой блок и заведён. Отказ громкий и одноразовый — описание чинится
навсегда, тогда как молчание длилось девять объявлений подряд.

    steps[0] (write_fix): max_tokens is a generation parameter and belongs
    inside `sampling: {max_tokens: 8000}`, not on the step itself

Ловушка — inline *Sampling в Run и в Profile под тегом schema:"-": парсер
читает эти ключи ровно для того, чтобы Validate их отверг, а тег говорит об
этом тесту, который держит схему равной структурам (описать их в схеме
значило бы утверждать обратное тому, что делает Validate).

Перечень полей и в проверке, и в тесте берётся рефлексией по типу: ровно
этот дефект — «параметр, которого движок не читает, исчезает молча», — и
список из семи имён руками воспроизвёл бы его для восьмого. Проверено
временным добавлением поля: подхватывается само.

Профиль закрыт тем же кодом. Живого случая там не нашлось, но именно в
профиле лежал работающий потолок — значит это та форма, с которой копируют,
а скопировать её уровнем выше и есть ровно та ошибка.

Формат не вырос, поэтому 2.4.1. Скилл, несущий такое поле, теперь не
грузится — с напечатанным исправлением; сам потолок и до этого не применялся,
меняется только враньё о нём.
@inhuman
inhuman merged commit 795aae8 into main Aug 15, 2026
1 check passed
@inhuman
inhuman deleted the step-sampling-fields branch August 15, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant