format: a CSV quotes the fields that need it, and you choose which - #46
Merged
Conversation
Adds quote_style to the csv format, taking minimal, all or none. The default is minimal, which changes the bytes of every table this tool writes - the description column used to be quoted on every row and is quoted now only when it carries the separator. This is a breaking change under D11 and it is deliberate. Measured on a 4 kB table at seed 7 before the change: 9 of 44 rows carry a description with no separator in it, because the phrase is three to seven words and drops a separator every third one. Those nine lose their quotes. Sizes are unchanged and every reader that took these files still takes them. The values are the RFC 4180 vocabulary and nothing outside it. A fourth name that preserved today's bytes was rejected: the release this belongs to closes with a major bump either way, so a clean vocabulary costs nothing now and a fourth name would have been carried forever. Two things are not obvious from the list of values. none changes the CONTENT, not only the punctuation. An unquoted field cannot hold a separator without ending early, so the description stops carrying one - in the phrase and in the padding both. A value that only removed the quotes would produce a ragged row at exactly the right size. And the closing row is built to the byte, so under minimal the decision to quote changes the length that the decision depends on. Measured over 59 sizes: the padding first carries a separator at 30 B of description, so the ambiguous band is two sizes per dialect. It is resolved by MEASURING - the quoted length is built first, and if it carries the separator the quotes are earned, otherwise the description is rebuilt to the full room with the separator withheld. A threshold constant was rejected as arithmetic that has to keep agreeing with the bytes beside it. The floor moves with the setting, the way the dialect already does: 115 B rather than 117 under minimal and none, 139 B under all. Twelve combinations, eight distinct floors, all measured with the binary. all quotes the header too, because a header is a row of fields and a writer told to quote everything quotes those as well. The alternative left the structural checker with an "except the first row" exception, which is where a defect hides. The checker is now told the style and judges it, with negative controls in both directions. That matters more here than for the other axes: every one of the three styles produces a well formed file, so nothing about the table gives the style away and a rubber stamp would have been silent. Verified: 1152 files swept over every size from 115 to 260 B across three styles and two delimiters - exact size, six columns under Python's csv module, quoting matching the style, and no needless quote under minimal. All 48 dialect combinations through the structural checker. LibreOffice Calc headless reads all three variants as six columns on every row, which closes D4 for the new values. 16 of 16 mutations caught. Two of those mutations were NOT CAUGHT at first, and the fault was in the guard: both only shift the ANNOUNCED floor, which is invisible elsewhere because Shortest is the worst draw and a real row sits some forty bytes under it. The only handle is the count of distinct floors, and it was written as six where the axes make eight. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…stry Two things, and the first is why CI was red. The coverage gate died on a limit nobody had chosen. Go allows ten minutes per package by default while that job allows twenty for all of it, so internal/guard hit the first without coming near the second - a stack trace out of whichever test was running when the alarm went off, instead of a failure naming something. Exactly what the race job above it met on 2026-08-25 and fixed the same way. Measured on the runner rather than guessed: the step took 375 s and 457 s on two consecutive main runs, which is 22 percent of variance on code that barely moved, and the default cuts in at 600 s. This branch added eight seconds of coverage instrumented work - measured, both new CSV guards together - and tipped it. Eight seconds is not what went wrong. 457 against 600 was never a margin. The second thing is O176: the README settings table had no guard and disagreed with the registry on seven rows. log said "none" while carrying eight settings, zip and targz listed three of eight, and avif and jxl had no row at all. The site has had this guard since it was built. The one page a visitor reads first did not. Two guards rather than one, because the table turned out to be the second half of the problem. The list at the top of the README was missing jxl outright - it arrived on 2026-08-31 as the twenty fourth format and never reached that list, so the page offered twenty three while the binary shipped twenty four. The prose said "twenty two" in three places and "24" in two: one file, three different numbers about one thing. Settings are compared as SETS. Whether a row reads "width, height" or the other way round is a question about English, and a guard answering it would be refusing prose rather than catching a lie. The count spelled out in words is deliberately not guarded. That was measured and rejected on 2026-08-05 in copiednumbers_test.go: the general form raised 43 findings and most were false, because "24 formats" and "25 formats" and "150 formats" answer three different questions here. Both guards are proven by mutation - a setting renamed in the registry, and a format registered under a name the list does not carry. That matters because the neighbouring TestTheFormatDocumentAgreesWithTheRegistry sits on notProvenByMutation, and that list is only allowed to shrink. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t that way Follow through on the commit before this one, which fixed the coverage gate and left three jobs sitting in exactly the same place. Measured on the green run of 2026-09-03, rather than assumed from the one job that went red: the test step takes 399 s on ubuntu, 476 s on windows and 491 s on macOS, all against Go's unstated ten minutes a package. macOS had 109 s of room. The same fleet was measured swinging from 479 s to past 600 s between two runs of one branch, so the margin was smaller than the variance on every one of them. The release workflow had the same gap, and there a timeout would read as a red tree and stop a release that was fine. Six whole tree runs now state a timeout. The two that already did are unchanged. The guard is the point of this commit rather than the flags. This is the SECOND time the project has lost a run to Go's default - the race detector met it on 2026-08-25 and the answer was a long comment beside that one job, which is why the coverage gate met it again eight days later. A diagnosis recorded at one step does not protect the next one, so the reasoning has moved out of the comments and into TestEveryWholeTreeTestRunStatesItsOwnTimeout. Proven by mutation. It reads the workflows through the YAML parser rather than as text, because a run block can be folded and the flag would then sit on a different line from the command. Only whole tree runs are asked: a targeted -run walks a handful of tests, and the fuzz step carries -fuzztime, which is its own budget. One stale claim fixed on the way. The matrix job's own comment said "the matrix runs in about a minute", which had not been true for a long time - it is eight, and the numbers are written down now instead of a word. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
quote_styleto thecsvformat, takingminimal,allornone.This is a breaking change under D11 and it is deliberate. The default is
minimal, so the description column - quoted on every row until now - isquoted only when it carries the separator. Measured on a 4 kB table at seed 7
before the change: 9 of 44 rows carry a description with no separator,
because the phrase is three to seven words and drops a separator every third
one. Those nine lose their quotes. Sizes are unchanged and every reader that
took these files still takes them.
The values are the RFC 4180 vocabulary and nothing outside it. A fourth name
preserving today's bytes was rejected: the release this belongs to closes with
a major bump either way, so a clean vocabulary costs nothing now and a fourth
name would have been carried forever.
Two things that are not obvious from the list of values
nonechanges the CONTENT, not only the punctuation. An unquoted fieldcannot hold a separator without ending early, so the description stops carrying
one - in the phrase and in the padding both. A value that only removed the
quotes would produce a ragged row at exactly the right size, which is the
defect this tool offers deliberately elsewhere.
The closing row is built to the byte, so under
minimalthe decision toquote changes the length that the decision depends on. Measured over 59
sizes: the padding first carries a separator at 30 B of description, so the
ambiguous band is two sizes per dialect. Resolved by MEASURING - the quoted
length is built first, and if it carries the separator the quotes are earned,
otherwise the description is rebuilt to the full room with the separator
withheld. A threshold constant was rejected as arithmetic that would have to
keep agreeing with the bytes beside it.
The rest
The floor moves with the setting, the way the dialect already does: 115 B
rather than 117 under
minimalandnone, 139 B underall. Twelvecombinations, eight distinct floors, all measured with the binary.
allquotes the header too, because a header is a row of fields and a writertold to quote everything quotes those as well. Leaving it bare would have left
the structural checker with an "except the first row" exception, which is where
a defect hides.
The structural checker is now told the style and judges it, with negative
controls in both directions. That matters more here than for the other axes:
all three styles produce a well formed file, so nothing about the table gives
the style away and a rubber stamp would have been silent.
Verified
and two delimiters - exact size, six columns under Python's
csvmodule,quoting matching the style, no needless quote under
minimal.negative controls.
every row, which closes
D4for the new values. A spreadsheet rather than aparser, so it answers a question our oracles cannot.
Two of those mutations came back
NOT CAUGHTat first, and the fault was inthe guard rather than the code: both only shift the ANNOUNCED floor, which is
invisible everywhere else because
Shortestis the worst draw and a real rowsits some forty bytes under it. The only handle is the count of distinct
floors, and it was written as six where the axes make eight.
An earlier version of the dialect guard also asserted that the closing row
always carries a separator inside its quotes. Under
allit need not, becauseallquotes regardless - the guard assumed a state it does not always reach.It now walks a band of 24 sizes and asserts it reached one.
Full suite,
preflight --quick(12 of 12, including the three CI-only checks)and pinned
staticcheck v0.8.1are all green locally.🤖 Generated with Claude Code