A generation parameter written on a step is refused, not dropped - #14
Merged
Conversation
The schema is a second copy of what the format is, and nothing kept it equal to the first: max_tool_errors is read on every step and was described nowhere. A host that repairs a description against the schema then drops a legal field, quietly. The test compares yaml tags with the schema's properties, so the next field added to one of them fails here rather than in somebody's skill.
kubectl_get answers {"items": [...]}, the skill said for_each in: pods, and the value - multi-line and not starting with [ - was split BY LINES. The first item was the line {, and the failure surfaced a step deeper, on a field access, where nothing points back at the loop.
An object with one list is now iterated as that list and the step is marked degraded: guessing for the author has to be audible. An object with several is refused, naming the fields and the author's own variable in the example.
max_tokens на уровне шага YAML выбрасывал молча: потолок объявлен и
одновременно отсутствует, а на глаз это неотличимо от шага, который потолка
не просил. Замер по живому каталогу из 29 скиллов: из десяти объявлений
потолка девять написаны на шаге и не действовали ни дня; единственное
работающее лежало в sampling профиля.
Это не невнимательность одного автора. Те файлы писал руками человек,
знающий формат, и сборщик скиллов генерирует ту же форму: остальные
параметры шага (model, tools, max_calls, response_schema, one_of) пишутся
именно на шаге, и ручка сэмплирования читается как их сосед.
Из двух вариантов выбран отказ, а не слияние в sampling: движок держит
параметры генерации отдельным блоком намеренно, и алиас размывает границу,
ради которой блок и заведён. Отказ громкий и одноразовый — описание чинится
навсегда, тогда как молчание длилось девять объявлений подряд.
steps[0] (write_fix): max_tokens is a generation parameter and belongs
inside `sampling: {max_tokens: 8000}`, not on the step itself
Ловушка — inline *Sampling в Run и в Profile под тегом schema:"-": парсер
читает эти ключи ровно для того, чтобы Validate их отверг, а тег говорит об
этом тесту, который держит схему равной структурам (описать их в схеме
значило бы утверждать обратное тому, что делает Validate).
Перечень полей и в проверке, и в тесте берётся рефлексией по типу: ровно
этот дефект — «параметр, которого движок не читает, исчезает молча», — и
список из семи имён руками воспроизвёл бы его для восьмого. Проверено
временным добавлением поля: подхватывается само.
Профиль закрыт тем же кодом. Живого случая там не нашлось, но именно в
профиле лежал работающий потолок — значит это та форма, с которой копируют,
а скопировать её уровнем выше и есть ровно та ошибка.
Формат не вырос, поэтому 2.4.1. Скилл, несущий такое поле, теперь не
грузится — с напечатанным исправлением; сам потолок и до этого не применялся,
меняется только враньё о нём.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
max_tokenslives inSampling. Written at STEP level it was not read — YAMLdrops an unknown key without a word:
A ceiling declared and absent at the same time, and by eye that is
indistinguishable from a step that never asked for one.
The measurement
Over a live catalogue of 29 skills there were ten declarations of a token
ceiling. Nine were written on the step and had never worked a single day.
The one that worked sat inside a profile's
sampling.Not one author's inattention: those files were written by hand by someone who
knew the format, and a skill-writing model produces the same shape. Every other
parameter of a step —
model,tools,max_calls,response_schema,one_of— is written on the step, so a sampling knob reads as their neighbour.What this does
Refuses it, with the block and the author's own values printed:
Refused rather than folded into
samplingas an alias: the engine keepsgeneration parameters in a block of their own on purpose, and an alias blurs the
line that block exists to draw. A refusal is loud and paid once — a description
is fixed for good, where the silence lasted nine declarations in a row.
The values in the suggestion are the author's, deliberately. The previous
diagnosis of this — from a host repairing descriptions against the schema — said
"an unknown key was removed", about a key that is perfectly known, just one
level down. An author told their key is unknown goes looking for a typo, not for
a block.
How it is caught
An inline
*SamplingonRunand onProfile, taggedschema:"-": the parserreads these keys in order for
Validateto refuse them, and the tag says so tothe test that keeps the schema equal to the structs — describing them in the
schema would state the opposite of what
Validatedoes.Both the check and its test enumerate the fields by reflection over the type.
That is the point rather than a convenience: the defect is that a parameter the
engine does not read disappears in silence, and a hand-written list of seven
names would recreate it for the eighth. Verified by temporarily adding a field —
the case and the refusal both appeared with no edit.
Two things beyond the letter of the task
is where the ceiling that DID work was written, so it is the shape an author
copies from — and copying it a level too high is exactly this mistake.
nothing but a stray
max_tokensgets the diagnosis it needs instead of aformally correct and useless one.
Version
2.4.1 — the format gained no field, so by this repository's convention it is
a patch, with the precedent of 2.2.4 ("Fixed, and a behaviour change").
The ceiling was not being applied before either, so nothing it did changes —
only the lie about it does.