Parse 3-D references (sheet ranges) - #51
Open
gthb wants to merge 62 commits into
Open
Conversation
"Jan:Dec!A1" is a 3-D reference to sheets "Jan" through "Dec", but JAN and DEC are also valid column letters, so the A1 lexer claimed "Jan:Dec" as a column beam and left a bare "!" behind — which parse() then rejected as an unknown operator. Same for "A:C!A1", and for "C1:C5!R1C1" in R1C1 mode where C1 and C5 are valid column parts. Excel forbids ":" in sheet names, so a colon ahead of the "!" can only separate two sheet names. Reject "!" as a beam terminator (canEndBeam, mirroring the canEndPartialRange rule for ternary ranges), and in R1C1 mode also bail out of the range lexer entirely when a complete "X:Y" pair is followed by "!", so the context lexer sees the whole prefix. The context lexer does not yet accept the colon, so for now these inputs lex as a name, a ":" operator and a reference. That is what "fool:bard!A1:B2" already did, and no longer throws.
"Sheet1:Sheet2!A1" refers to cell A1 on every sheet from Sheet1 to Sheet2. The quoted spelling already worked, because the quoted-context lexer accepts any character between the quotes; the unquoted one stopped at the colon, so "fool:bard!A1:B2" came out as a name, a ":" operator and a reference, and parseA1Ref rejected it. Admit a single colon into an unquoted context, in both the standalone context lexer and the combined name/function/context lexer. It may not lead the context and must have a name on either side of it, and only one is allowed, since Excel forbids ":" in sheet names and a sheet range has exactly two endpoints. A colon inside brackets stays part of the workbook name, as before. The colon is the first character that can end the name run while the context run carries on, so the name/function/context lexer now remembers where the name ended and falls back to it when the context turns out not to be one: "foo:B2" still lexes as a name, a ":" operator and a reference. The sheet range lands in the sheet slot as a single compound name — context: [ 'Sheet1:Sheet2' ], or sheetName: 'Sheet1:Sheet2' in xlsx mode — which is the shape the quoted spelling already produced. Serialization needs no change: ":" is a banned character, so the prefix is quoted on output. Sheet order is left alone. Whether "Sheet2:Sheet1" should read "Sheet1:Sheet2" depends on the order the sheets appear in the workbook, which is not knowable from a formula string.
Cover both the context and the xlsx variants of tokenize, parseA1Ref,
stringifyA1Ref, parseR1C1Ref, stringifyR1C1Ref, parseStructRef,
stringifyStructRef, fixFormulaRanges and the A1/R1C1 translations, plus
addTokenMeta grouping.
Includes regression guards for the constructs a sheet range is easily
confused with: cross-sheet ranges ("B!F2:B!F20", "Sheet1!A1:Sheet2!B2")
stay two references, "foo:B2" stays a range operator between a name and
a cell, and "Jan:Dec" without a "!" stays a column beam.
Excel writes a 3-D prefix bare when neither of its sheet names needs
quoting — its own documented example is =SUM(Sales:Marketing!B3), and
that is what both the formula bar and the stored formula show. Quoting
it regardless is legal but loses fidelity: a consumer that reads
SUM(Jan:Dec!A1) and writes it back would emit SUM('Jan:Dec'!A1).
The colon separates two names rather than belonging to either, so
needQuotesSheet splits the sheet scope on it and applies the existing
rules to each half, quoting the whole prefix as one unit if either half
calls for it. That is the same whole-prefix convention stringifyPrefix
already uses for '[My File.xlsx]Sheet1'!A1.
Jan:Dec!A1 bare
Sales:Marketing!B3 bare
[Book.xlsx]Sheet1:Sheet2!A1 bare
'Sheet1:Sheet 2'!A1 one half has a space
'1:5'!A1 digit-leading halves
'A1:B2'!A1 range-like halves
'A:C'!A1 "C" alone reads as an R1C1 column
'[1]Sheet1:Sheet2'!A1 the workbook name forces it
Only the sheet slot is split. A colon can reach a path scope on its own
as a Windows drive letter without dividing it into two names, and a
workbook name is not a sheet name either, so both keep using needQuotes.
Excel resolves "=SUM(A1:B2!C3)" as cell A1 joined to 'B2'!C3 — it stores
the formula that way and the result is #VALUE!. A column-shaped left
side does not lose out like that: "=SUM(A:C!A1)" and "=SUM(Jan:Mar!A1)"
are both sheet ranges. To reference sheets named A1 and B2 the prefix
must be quoted: "=SUM('A1:B2'!C3)".
The A1 lexer claimed "A1:B2" whole and left a bare "!" behind, which
parse() rejected as an unknown operator. Fold the "!" rejection into
canEndRange, which no range may end at, so the range ends at "A1" and
"B2!" is left to prefix the reference on the right. The column-shaped
spellings are unaffected: they end at the column, which is not a range
by itself, so the whole sheet range falls to the context lexer.
This subsumes canEndBeam, added a few commits ago for the same reason,
and the standalone "!" check in the single-cell branch.
Also pin the one spelling Excel refuses outright: "$" on an unquoted
sheet name ("=SUM($Jan:$Mar!A1)" is rejected on entry, and a file
holding one does not open at all). fx already declines to read it as a
context, "$" not being a context character; keep it that way. "$" is
legal in a sheet name, so "'$Jan:$Mar'!A1" is a valid sheet range.
Excel accepts "=SUM([Book.xlsx]S1:S3!A1)" on entry but normalizes it to
"=SUM('[Book.xlsx]S1:S3'!A1)", adding quotes that neither sheet name
calls for. It goes the other way for a single sheet, where needless
quotes are instead removed: "'[Book.xlsx]Sheet1'!A1" becomes
"[Book.xlsx]Sheet1!A1".
So the per-endpoint rule governs only an unqualified sheet range
("Jan:Dec!A1" bare, "'Sheet 1:Sheet 2'!A1" quoted). Once a workbook or
path scope is present, quote the whole prefix regardless.
In a saved xlsx the prefix holds the external-link index rather than a
path, "'[1]S1:S3'!A1", which the same rule covers.
"=foo:'bar'!A1" lexed as the beam "foo:" followed by a separate reference "'bar'!A1", and normalized to "=A:FOObar!A1" — silent corruption of the kind this branch exists to remove. "='foo':'bar'!A1" fared no better, and "='foo bar':'baz'!A1" shredded into unknown tokens. The beam guard added earlier only recognized an unquoted sheet prefix. A quote opens one just as much as a name followed by "!", so canEndRange now rejects it too; the open-ended beam was the tell, the lexer having bailed at the quote with nothing past the colon. That alone would leave these as a name, a ":" operator and a reference, so the context lexers now read the far end of a sheet range whether it is quoted or bare, and the near end likewise. Unquoting moves from the whole prefix to each quoted run within it, which leaves a wholly quoted or wholly bare prefix exactly as it was. foo:'bar'!A1 → foo:bar!A1 'foo':bar!A1 → foo:bar!A1 'foo':'bar'!A1 → foo:bar!A1 'foo':'bar baz'!A1 → 'foo:bar baz'!A1 'foo bar':'baz'!A1 → 'foo bar:baz'!A1 '[Book.xlsx]foo':'bar'!A1 → '[Book.xlsx]foo:bar'!A1 Excel normalizes all of these away on entry, so they never occur in a file it wrote. This is input tolerance for hand-written and third-party formulas, not round-trip fidelity, and the output settles on the spelling Excel would have written. A trailing colon with nothing after it stays an open-ended beam, so "SUM(foo:)" is untouched.
A 3-D reference puts both sheet names in the sheet slot, "Jan:Dec" where an ordinary reference has "Sheet1", and nothing but the colon inside the name tells them apart. Code that resolves a sheet name against a workbook and is handed the slot whole matches no sheet at all — and since a lookup finding nothing usually reads as "no such sheet" rather than as an error, it does so silently. The shape is the right one and changing it is a major-version matter, so give the hazard a supported way out instead: splitSheetRange returns the two sheet names, or undefined for an ordinary single-sheet scope. It was already there, private, doing this job for the quoting rules. Moved out of stringifyPrefix.ts, whose subject it no longer is, into its own module, and documented — including that it takes the sheet scope alone, a path scope being free to hold a colon of its own in a Windows drive letter. The parseA1Ref docs point at it, that being where a consumer lands.
"Only the former is a single reference" pointed ambiguously at one of two things named in the previous sentence, and was wrong either way: a plain A1:B2 is one REF_RANGE token and one ReferenceIdentifier too, so being a single reference is not what separates a 3-D reference from a cross-sheet range. isReferenceNode is strictly node.type === REFERENCE and agrees. Name both sides and give the observable difference: Jan:Dec!A1 is one token parsing to one ReferenceIdentifier, which parseA1Ref resolves; Sheet1!A1:Sheet2!B2 is three tokens parsing to a BinaryExpression over two ReferenceIdentifier nodes, which parseA1Ref returns undefined for. Verified against the code. Also warn, in the section a reader of this would be in, that the sheet slot may hold two names and that resolving one against a workbook needs splitSheetRange first — with the failure mode stated, since it is a silent one. Say outright that Excel normalizes a sheet range on entry and how: it orders the two ends, collapses a degenerate range, and corrects the case of each end to the sheet's own. fx does none of the three because each needs the workbook's list of sheets, which formula text does not carry. Saying so turns an apparent gap into a stated boundary, and settles that the un-normalized spellings are valid input. Drive-by: "endpoint" throughout the section is now "end", the ends of a sheet range not being the corners of a range; and a duplicated note about colons in path scopes is gone.
What Fx lacks is the workbook's sheet names in order, not its sheets: ordering the two ends, spotting a degenerate range, and correcting each end's case are all name comparisons.
The dash there had been written as a literal \u2014 escape rather than the character it stood for.
The note claimed a colon inside a context can only separate a sheet range, which is not true of a context generally: a quoted one may hold a Windows path, whose drive letter carries a colon of its own. It is true of this table, which lexes unquoted contexts, where a path cannot appear at all.
`parseA1Ref('A1:B2!C3')` read the prefix as a sheet range and answered
`'A1:B2'!C3`, a reference into sheets named `A1` and `B2`. Excel, the
tokenizer and the parser all read the same text as cell `A1` joined to
`'B2'!C3`, so the answer silently changed the formula's meaning; before
sheet ranges were parsed at all, it was `undefined`.
The formula lexers settle this by running `lexRange` ahead of the context
lexers at each position, so a cell-shaped run wins before a context can
claim it. The reference lexer set runs `lexContextUnquoted` first, and it
had no equivalent test, so `lexContextUnquoted` now asks `lexRange`
whether the run up to the colon is a whole range and declines the colon
if it is.
The check sits behind the "only one colon, not leading" test, and the
formula lexers reach `lexContextUnquoted` only for a context starting
with a digit or a dot, which can never be cell-shaped, so tokenizing is
unaffected.
A quote never ends a beam: "canEndRange" rejects it outright, so a beam that runs into one is not lexed as a beam at all, whether or not a sheet prefix follows. What the cases guard is that the beams and quoted prefixes that have nothing to do with sheet ranges still lex as before.
That the range of a 3-D reference is normalized but its sheet range never is, is behaviour a caller needs to know, and a free-floating file comment never reaches the generated docs. Move the statement of it into the fixTokenRanges docstring, beside the enumeration of what does get normalized, and leave the file comment holding the three Excel normalizations it stands in for.
`parseA1Ref` takes only `allowNamed` and `allowTernary`, so both snippets
in Prefixes.md passing `{ xlsx: true }` were teaching an option that does
not exist: a reader copying either gets the context variant back, with
the option silently ignored. The xlsx counterparts live behind the
`@borgar/fx/xlsx` entry point, so say that once and drop the option.
The `[1]!A1` snippet predates 3-D references and was already wrong; it is
fixed alongside the new one rather than left as the odd one out.
"canEndRange" refuses "'" whether or not a sheet prefix follows, so an unfinished "=A1'" now lexes as one UNKNOWN token where it used to be a range and a stray quote. That is a deliberate choice rather than a consequence nobody looked at, but nothing said so or held it in place, so a later reader would have taken it for an overreach and narrowed it. Note the reasoning where the rejection lives, and cover both the widened spellings and "=A1:'Sheet 2'!B2", the half-typed formula that would actually suffer from a narrower rule and does not, since it reaches its range through the colon.
Returning early for "foo:'bar'!A1" leaves the loop before the mask bug can swallow the string, so a sheet range whose far end is quoted now lexes even when its near end starts above U+00B4, while the same range unquoted and even a plain reference off such a sheet still do not. An unexplained improvement in one spelling out of three invites reading the other two as the intended behaviour, so say where it comes from and that the mask is fixed elsewhere.
The guard that hands "C1:C5!R1C1" to the context lexer fired on any second R1C1 part, so "R1C1:R2C2!R3C3" and "RC:R2C2!R3C3" became sheet ranges as well. Both have a complete cell on the left, which the A1 lexer gives to the range operator, and which master gave to it here too. Exempt a left side that has both a row and a column part, so the two notations read such a pair the same way and only the quoted spelling, "'R1C1:R2C2'!R3C3", is a sheet range.
The colon of a sheet range had to be preceded by something, but not by a sheet name, so "[Book.xlsx]:Sheet2!A1" read as a reference to a sheet called ":Sheet2" — a name Excel cannot produce, and one master rejected outright. Measure the near end from the workbook brackets rather than from the start of the whole prefix, which is where the sheet name begins. The tests for this rule went with what they name only when a beam or an operator settled the spelling first; they now also cover names that are not column letters and so reach the context lexer.
Nothing failed when advQuotedSheetName stopped refusing brackets, and the spelling it guards is one Excel writes: when the far end of a sheet range names no sheet, Excel manufactures an external link for that name and stores "Jan:'[1]Nope'!A1", which is a reference to another workbook rather than a sheet range.
Excel does not normalize "plain:'has space'!A1" into a sheet range; it keeps the text as typed and evaluates it to #NAME?, so a separately-quoted pair is never a sheet range to Excel and the shape does not reach fx by way of Excel's normalizer. Accepting it is a leniency towards other producers. Excel does write the shape, but for a different thing: a sheet range that lost an end keeps the colon and quotes whatever survives, "Nope:'Mar'!A1" for a missing first end and "Jan:'[1]Nope'!A1", with a manufactured external link, for a missing second. Prefixes.md now says so, since a caller reading "foo:bar" out of the first one should know what it is looking at.
A colon ahead of the "!" is unambiguous only inside the sheet scope. The rest of the prefix can hold one that means something else — a Windows drive letter in a path, or a colon in a workbook file name — which is what the paragraph about passing splitSheetRange only the sheet scope is about.
Its @returns line offered "a single sheet name" as the only alternative to a sheet range, but ":x", "x:" and "a:b:c" are also undefined and are none of those. Drive-by: a sentence in fixRanges that read "normalizes one three ways".
The rule was known only from "A:C" gaining quotes while "A:AB" stayed bare, and fx read that as per-end quoting without saying so. Measured in Excel on a workbook with sheets A, B, C, D, AA and AB, the trigger is the name: "B:C" and "C:D" are quoted, "A:B", "AA:AB" and "A:AB" are not, so neither the length of the names nor the number of sheets spanned matters, and "C" forces the quotes from either end. That is what needQuotes already yields, being applied per end over a pattern covering R, C, RC and cell-shaped names. Pin the two ends of the measured set that were untested, "B:C" and "AA:AB", and say the rule in Prefixes.md and at needQuotesSheet as a rule about names.
"A1:B2!R3C3" in R1C1 mode is the sheet range A1:B2, where A1 notation reads "A1:B2!C3" as a range operator joining A1 to 'B2'!C3. The asymmetry looks like a bug and is Excel's: with the R1C1 reference style on, "=SUM(A1:B2!R3C3)" sums R3C3 across sheets A1 through B2 while the A1-notation spelling is #VALUE!. Which names are cell-shaped is decided in the notation the cell part uses, and the stored <f> is A1 notation either way, so the same two sheet names read oppositely depending on how the formula is written. fx already does this. Pin it beside the A1-mode case and say so in Prefixes.md, where a reader meeting the A1 rule is likeliest to assume the notation makes no difference.
lexContext already imports lexRange, so a range lexer reaching back for advSheetName would close an import cycle. Move the two scanners and the character class they share to a leaf module instead. No behaviour change.
"C1:C5!R1C1" was a sheet range only because "C5" happens to lex as an R1C1 column part; "C1:Dec!R1C1", "C1:'Dec'!R1C1", "C:D!R1C1" and "R1:Total!R1C1" all lost their far end and left the near one standing alone as a beam. The far end of a sheet range is a sheet name, so whether it also looks like an R1C1 part has nothing to do with it, and the A1 lexer already reads the counterpart spellings as sheet ranges. The fallout was not confined to reading R1C1: translating "=SUM(C:D!A1)" to R1C1 carries the sheet range through verbatim, so translating the result back yielded "=SUM(C:C:'D'!A1)" — a different reference. Look ahead over a sheet name from the range operator, as the context lexer does, alongside the existing check on a far end that did lex as an R1C1 part. Both are needed: an R1C1 part may run past what a sheet name may hold, as the brackets of "C1:R[1]C[1]" do.
The note about "$" and the one about the guard below it were folded into one paragraph, where the first two lines describe a rule the code under them does not enforce.
It hands its options straight to lexContextUnquoted, which now reads
r1c1, mergeRefs and allowTernary off them to ask lexRange whether a
sheet range is a range instead. The declared "{ xlsx: boolean }" hid
that, and structural typing let it through because the three are
optional; getTokens passes one shared object, so it worked by luck.
Mutation-testing every guard on the branch left these four able to be removed
with the suite still green:
- a quoted endpoint that never closes, and a closed one with no prefix behind
it, name no sheet: "Jan:'Dec!A1" and "Jan:'Dec'"
- a colon inside the workbook brackets is a Windows drive letter and divides no
sheet names: "[C:\Book.xlsx]Jan:Dec!A1"
- the sheet is what follows those brackets, so a workbook ahead of a sheet range
neither hides a cell-shaped near end nor becomes one: "[1]RC:Dec!A1"
- the reference lexers reach the quoted far end by a path the formula lexers
cannot, a leading digit being a number there: parseA1Ref("1:'Dec'!A1")
A near end shaped like a cell keeps the colon for the range operator, which endsAWholeRange settles by asking whether a range ends exactly at that colon. A ternary range can reach past it, though — "R1C1:C5" in "R1C1:C5.b!R1C1", where the "." carries the far end's name on past what the range grammar takes. Reaching past the colon means the range operator has already claimed it, so that is no less an answer of "yes" than landing on it; only stopping short is "no". parseR1C1Ref read the pair as the sheet range "R1C1:C5.b" instead. Needs allowTernary, which is off by default.
- canEndRange no longer settles "Jan:Dec!A1" or "foo:'bar'!A1": lexRange stands aside for a sheet range before either range lexer runs, so what is left to this function is the cell-shaped near end. Removing its "!" and "'" tests now changes only "A1:B2!C3" and "=A1'". - A bare prefix may hold brackets — "[Book.xlsx]Sheet1!A1" does. What brackets cannot do is reach either end of a sheet range. - unquoteParts does not leave a wholly quoted prefix untouched; it unquotes it, as it always did. - A name holding a high character does lex, into one name token that swallows what follows. It is the token that is wrong, not the lexing that is missing. - parseA1Ref takes a reference string, not a formula. Also spells the soft hyphen and U+FFFF of a character class as escapes rather than as the characters themselves, which are invisible or unrenderable in an editor, and rewraps a docstring line past 100 columns.
The loop that walks a name, a function call or an unquoted context looked its per-character mask up with `s`, the token's first character, where every other use of it means `c`, the one being read. A name starting above the table's last entry therefore made every character after it look high too, "!" included, so "Ærið!A1" came back as one name token and the prefix inside it was never seen. The reference lexers reach a context by another route and read the same string as a prefix and a range, so the two disagreed on anything with a high-character sheet name. The table covers up to U+00B3 and the guard admitted only what is past U+00B4, leaving that one character in neither. It is a name character by the grammar the table is built from, so the guard now takes it. Found by the property that the formula lexers and the reference lexers must agree on the prefix.
A range lexer stops wherever its own grammar runs out, which can be part-way through a sheet name, because "." is the one character a sheet name may hold that a range is also allowed to end on. "v1.0!A1" was therefore lexed as the cell V1, a stray ".", a "0" and a "!", while the reference parsers read the whole "v1.0" as the sheet. A sheet range already had a guard for this; a lone sheet name did not. startsSheetRange becomes startsSheetPrefix and answers for both: a name that reaches the "!" whole is a prefix, whether a second name and a colon stand between it and the "!" or not. A lone name needs no cell-shape test — "!" is no range character, so no range lexer can take one whole in the first place. Found by the property that the formula lexers and the reference lexers must agree on the prefix.
"." is a sheet-name character and Excel does not ban it from a sheet name, so ".:Dec!A1" is a sheet range over sheets "." to "Dec". Both lexer sets read the trim range operators ahead of everything else, so both took the ".:" and left "Dec!A1" behind — a trim operator with nothing on its left, which is a reading of nothing. They now stand aside for a sheet prefix as the range lexers do. The test costs nothing where it is not needed: only a leading "." can open both a trim operator and a sheet name. Found by the round-trip property, which reached the spelling by writing back a sheet range that stringifyPrefix had no reason to quote.
Where the colon is followed straight by "!" the second sheet name is missing, which lexContextUnquoted noted and then went on scanning for a later "!" to close the prefix at. That let the first "!" be taken for the far end's name: "a:!!A1" came back as a sheet range over a sheet named "!". "!" is not a sheet-name character, so no later "!" can supply what the colon lacks, and the prefix ends there. lexNameFuncCntx already stopped at that point. Found by the property that the formula lexers and the reference lexers must agree on the prefix.
Bare "TRUE!A1" lexes as the boolean joined to a reference, so a sheet named TRUE or FALSE has to be quoted for its prefix to read back at all — as much as a range-like or a digit-leading name does, both of which needQuotes already covers. This bites at either end of a sheet range as well as on its own, and there it is a regression: a sheet range is quoted per endpoint, so "'False:Jan'!A1" was being written back as "False:Jan!A1", where the whole-scope quoting it replaced had held the pair together by accident, ":" being a banned character. Found by the property that a round trip keeps both sheet names.
Every review pass over this branch found something its predecessor's instruments could not see, and the corpus was the limit each time: the last one found two of the four faces of its main defect only after the corpus was extended by hand. So the remaining risk is inputs nobody thought to write down, which is what a generator addresses and another list of spellings does not. Three properties, each the invariant that actually caught something: - the formula lexers and the reference lexers agree on the prefix. This branch decides "is this a sheet range?" in two places with opposite lexer precedence, so any input they read differently is a bug in one of them - parse -> stringify -> parse reaches a fixpoint with both sheet names intact, through parseA1Ref/parseR1C1Ref (± xlsx), fixFormulaRanges, and both translators at four anchors. A writer emitting text its own reader takes for a different reference is the recurring failure here - the tokens of an input rejoin to exactly that input Sheet names are drawn from the whole legal character set — "." and "$" and "'", the R1C1 shorthand letters, characters either side of both boundaries of the context lexer's table — and shaped as cells, columns, R1C1 parts, digits and booleans, at each end of a span independently. Prefixes are bare, quoted whole, quoted per end, workbook- and path-qualified, and degenerate; cell parts cover both notations with beams, ternary ranges and trim operators; and each case draws its own combination of xlsx, mergeRefs, allowTernary, allowNamed and r1c1. Failures are shrunk before they are reported, and the seeds fix what runs, so a failure in the suite is reproducible from the line that reports it. The suite runs a slice of a few thousand cases; scripts/propertySweep.ts runs as many as you have patience for and is the thing to reach for after touching a lexer or the sheet-name grammar.
Excel forbids ":" in a sheet name, which splitSheetRange already relies on, but advSheetName let a separately-quoted endpoint hold one: "RC:':'!A1" lexed as a sheet range over sheets "RC" and ":". splitSheetRange then refused to divide the scope, leaving a "sheet name" of "RC::" that names nothing and that the translators do not know to quote — so the same string read as a sheet range in A1 notation and as cell RC joined to ':'!A1 in R1C1. The colon that divides the two ends is behind us by the time a quoted endpoint is measured, so a second one there can only be a character in a name, and there are none. A colon inside a prefix quoted as a whole is the divider and does not come this way. Found by the property that a translation to R1C1 and back keeps both sheet names, 2.2 million cases in.
A name reading as a boolean is quoted, and a separately-quoted end may hold no colon. Prefixes.md is where the quoting rules are written out, and both are rules a caller can see from outside.
`R` and `C` were both measured; `RC` was not, so it is stated as what _Fx_ does rather than as Excel ground truth.
lexContextUnquoted runs ahead of lexRange on the reference path, so it asks lexRange whether a cell-shaped left end has already claimed the colon. That question is the one expensive test in the lexer, and it was being asked of every colon reaching it — including the range operator of an ordinary "A1:B2", where no prefix is in play at all. A CONTEXT token only ever leaves this lexer through the "!" branch, so without a "!" ahead there is nothing to settle and nothing to return. parseA1Ref over the 1000 references of benchmark/refs.json, none of which holds a "!", goes from 1610 to 1975 ops/sec.
Excel quotes a workbook- or path-qualified sheet range as a whole when the formula is entered, but it does not hold every stored formula to that: a hand-written <f> of "SUM([1]One:Three!A1)" survives a resave unquoted, and a sheet range that has lost an end is written with per-end quotes, "[ExtSrc.xlsx]zz top:'S3'!A1". Stating the rule as an unconditional "always" credits Excel with an invariant it does not keep, and reads as though the bare spelling were malformed input. The output rule is unchanged: Fx writes the quoted form, and reads either.
Refusing a bracket inside one end of a sheet range is Fx's own rule, not a statement about Excel: Excel writes "Jan:'[1]Nope'!A1" for a sheet range whose far end has stopped naming a sheet, binding the lost end to a manufactured external link. Prefixes.md and the pull request already say so; only this comment read as a rule of the formula language.
The round-trip property took a parser returning "unknown" and a serializer taking "never", and cast between the two at both call sites. Making the helper generic in the reference type lets each of the four pairings infer its own, so a parser handed the wrong serializer stops type-checking rather than being cast past.
The comments and docs added for 3-D references leaned on a handful of
indirect verbs where an ordinary one says the same thing, and several of
them hid which actor was doing the acting:
- "hold"/"carry" for a container relationship become "contain" or "have":
a sheet name contains a "."; a file containing "$Jan:$Mar!A1" does not
open; a path scope may contain a colon of its own.
- "shaped like a cell"/"cell-shaped" become "also a valid cell address",
and "the cell shapes of each notation" becomes "the cell addresses of
each notation". The identifiers (isCellShape, CELL_SHAPED, ...) are
unchanged, as are the test titles.
- "wins the colon"/"loses the colon" become "takes the colon"/"gives the
colon to the range operator".
- "stands aside" becomes "defers to" or "gives way to".
- "sits" becomes "is", "is in", or "appears".
- "reaches" is kept where it is exact ("reaches a fixpoint") and replaced
where it was vague: canEndRange never sees a whole sheet range, and the
half-typed "=A1:'Sheet 2'!B2" ends its range at the colon.
- "a leniency towards other producers" becomes a statement of who is
being lenient and why: Fx accepts the spelling to be forgiving of other
producers.
Naming the actor changes the claim for the better in two places. The
fixRanges docstrings said normalizing a sheet range needs sheet names
"which a formula does not carry" — it is Fx that does not know them, and
that is now what it says. The property module said Excel quotes both
shadowing spellings "so neither reaches a reader from a file Excel
wrote", now "so a reader never sees either of them".
docs/API.md is regenerated from the docstrings, not hand-edited.
The docs said to pass "only the sheet scope" but not in what form, so a
caller could not tell whether to hand over the raw, possibly-quoted sheet
prefix or an already-unquoted scope, nor whether the function unquotes
the names itself. Getting it wrong fails silently.
State the contract in all three places that describe it:
- splitSheetRange expects the unquoted scope, does no unquoting of its
own, and returns two names that need no further processing.
- parseA1Ref and parseR1C1Ref return that unquoted scope already, with
the surrounding quotes stripped and doubled apostrophes collapsed, so
every spelling of a prefix converges on one value: 'Sheet 1:Sheet 3'!A1,
foo:'bar baz'!A1 and 'It''s:Fine'!A1 yield the scopes Sheet 1:Sheet 3,
foo:bar baz and It's:Fine.
- Passing the raw quoted prefix is named as the trap it is:
splitSheetRange("'Sheet 1:Sheet 3'") returns two names with stray
quotes, with no error and no undefined to signal it, and a caller
matching those against a workbook's sheets matches nothing.
The @returns line also now says when undefined comes back: no colon, more
than one, or an empty half.
docs/API.md is regenerated from the docstrings, not hand-edited.
gthb
marked this pull request as ready for review
July 31, 2026 18:31
"Ærið:Ärger!A1" is one sheet range here and two endpoints in borgar#52, which carries the same mask fix without sheet ranges. Both assertions meet when borgar#52 lands and master merges in, and git reports no conflict because they sit in different parts of the file, so the failure would otherwise read as broken sheet-range lexing. The comment says which one survives.
That pass rewrote the comments and docs but left the titles, which are string literals, so a title could still say "a cell-shaped left side" beside a comment saying "also a valid cell address". The titles now use the same words as the prose around them: - "cell-shaped left side" -> "left side that is also a cell address" - "sheet names shaped like R1C1 parts" -> "that are also R1C1 parts" - "shaped like a cell in the other notation" -> "that are cell addresses in the other notation" - "a sheet name holding a ." -> "containing a ." - "carried through untouched" -> "passed through untouched" Names only: no assertion, input or expected value changes.
The claim that a colon in the sheet scope "can only be" separating two sheet names holds in front of a cell reference and nowhere else. Measured in Excel, a colon-bearing scope in front of a defined name or a structured reference is either the range operator or a workbook file name, and never a sheet range. Scope the claim and the splitSheetRange recipe accordingly, and record the five measured readings alongside the limitation they expose: Fx reads all four non-cell spellings as sheet ranges and splits them.
splitSheetRange and both parseA1Ref variants hand out a recipe for resolving the sheet slot without saying which references it holds for. Measured in Excel, it holds in front of a cell reference only: a colon-bearing scope in front of a defined name or a structured reference is the range operator when bare and a workbook file name when quoted. Scope the recipe and name the limitation, since Fx reads those spellings as sheet ranges and splits them regardless. docs/API.md regenerated.
A sheet range stands in front of a cell reference and nowhere else, so `Alpha:Gamma!SomeName` and `Alpha:Gamma!Table1[Col]` were being read as one reference where Excel reads two operands joined by the range operator. Measured in Excel, the discriminator is a rename. Renaming the sheet `Alpha` leaves `Alpha:Gamma!SomeName` untouched while both controls move — an ordinary `Alpha!SomeName` becomes `Bee!SomeName`, and a span over a cell becomes `Bee:Mar!A1` — so the `Alpha` in the subject is an ordinary name and not a sheet reference at all. In front of a table Excel goes further and rewrites `Alpha:Gamma!Table1[Col]` to `Alpha:Table1[Col]`, discarding the `Gamma!` as it discards any sheet prefix on a table; a span is not something that could be discarded from there. Add `spanTakesOperand`, which probes the range lexers at the operand, and consult it wherever a colon may join two sheet names: the sheet-range branch of `startsSheetPrefix`, both context lexers, and `lexNameFuncCntx`. The lone-name branch is left alone, so `Jan!SomeName` is untouched — only a colon raises the question. The serializers follow, or the round trip breaks: a sheet range is written per endpoint only in front of a cell reference, and in front of a name or a table the colon is quoted as the ordinary banned character it is. Without that half `stringifyA1Ref` would emit `a:b!Name`, which no longer parses back. The quoted spellings are deliberately left as they are. Excel reads `'Alpha:Gamma'!SomeName` as a workbook file name with no sheet component, and there is no way to represent that here.
Ten assertions pinned the belief that a span stands in front of an operand reached by name. That belief is wrong, so the assertions are what changes: each moves to the measured reading, and each says which of the two readings it is now asserting. The bare spellings become the range operation Excel reads them as — three tokens, a `BinaryExpression`, and `undefined` from the reference parsers — and the quoted spellings take over the sheet-range cases, those still being read as one scope. The serializer cases gain the quotes the colon now needs. No cell, range, beam or ternary assertion moves.
The note said Fx reads all four non-cell spellings as sheet ranges and splits each into two sheet names. That is now false for the bare ones, which read as Excel reads them, and no scope reaches `splitSheetRange` from there at all. What remains true is the quoted half: `'Alpha:Gamma'!SomeName` and `'Alpha:Gamma'!Table1[Col]` are still read as sheet ranges, Excel reading them as a workbook file name with no sheet component and there being no way to represent that here. Say that, and say why it is left for later. `docs/API.md` is regenerated from the docstrings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Parse 3-D references, where a prefix names a range of sheets rather than one sheet, as in
Sheet1:Sheet2!A1.Only the quoted spelling worked before, and by accident:
lexContextQuotedaccepts anything between the quotes, so'Sheet1:Sheet2'!A1parsed as an opaque sheet name that happened to contain a colon. Unquoted, it failed one of two ways depending on the names.fool:bard!A1parsed as aBinaryExpressionjoining the namefoolto the rangebard!A1, withparseA1Refreturningundefined.Jan:Dec!A1threwUnknown operator !, becauseJANandDECare valid column letters, soJan:Declexed as a beam and stranded the!.How
The sheet range occupies the sheet slot as a single compound name, so the unquoted spelling now produces what the quoted one always did:
context: [ 'Jan:Dec' ], orsheetName: 'Jan:Dec'in the xlsx variant. No type changes. In front of a cell reference nothing inside that slot needs disambiguating, since:is one of the characters Excel forbids in a sheet name — but that is the only place the sheet-range reading exists at all, which the Excel behaviour section takes up.One structural rule settles most cases: a range may not end where a sheet prefix begins, quoted or not. That makes
A1:B2end atA1inA1:B2!C3, leavingB2!to prefix the right-hand side. A single cell-address test instartsSheetPrefixanswers it for both notations — a near end looking like a cell address in the notation being read keeps the colon for the range operator, and anything else, matching the column-ID pattern or otherwise, leaves it to the sheet range. SoR1C1:R2C2!R3C3ends atR1C1in R1C1 notation, whileC1:C5!R1C1is a sheet range there. The formula lexers and the reference lexers reach that test from opposite ends and have to agree, which is why both asklexRangerather than deciding for themselves.Quoting is decided per end, quoting the whole prefix as one unit if either end calls for it — except that a workbook- or path-qualified sheet range is quoted whole on entry.
splitSheetRangeis exported because that slot is otherwise indistinguishable from an ordinary sheet name: anything resolving the sheet slot of a cell reference must split it first or silently match nothing. It takes the sheet scope alone, since a colon elsewhere in a prefix is a Windows drive letter or part of a workbook file name.Excel behaviour
Measured against Excel for Mac 16.113 except where noted. Where a sheet name is also valid as a cell address, the range operator wins:
SUM(A1:B2!C3)SUM(A1:'B2'!C3), evaluating to#VALUE!SUM('A1:B2'!C3)SUM(A:C!A1)SUM('A:C'!A1), a sheet range, quotes addedSUM(Jan:Mar!A1)SUM([Book.xlsx]S1:S3!A1)SUM('[Book.xlsx]S1:S3'!A1)Quoting is triggered by a sheet name, not by the sheet range: on a workbook with sheets
A,B,C,D,AAandAB, Excel storesSUM(A:B!A1),SUM(AA:AB!A1)andSUM(A:AB!A1)as typed and quotesSUM(B:C!A1)andSUM(C:D!A1).Cis one of the R1C1 shorthand names Excel quotes wherever they name a sheet, and it forces the quotes from either end, whatever the other end looks like;Rmeasures the same way, and Fx treatsRCalongside them. Neither the length of the names nor the number of sheets between them matters.Which names count as looking like cell addresses depends on which notation the formula is read in, so the same pair can go opposite ways. With Excel set to the R1C1 reference style,
=SUM(A1:B2!R3C3)is a sheet range summingR3C3across sheetsA1throughB2, where the A1-notation=SUM(A1:B2!C3)is#VALUE!. The inverse is what the same rule implies but is not measured: no probe named a sheetR1C1, so=SUM(R1C1:R2C2!R3C3)in R1C1 mode being cellR1C1joined to'R2C2'!R3C3is inference. Fx implements it symmetrically. This is a property of the notation a formula is read in rather than of the file: the stored<f>is A1 notation whichever style the interface displays, and in Fx it is ther1c1option on the call.Excel refuses
$on an unquoted sheet name,=SUM($Jan:$Mar!A1), and a file containing one does not open at all;'$Jan:$Mar'!A1is valid, since$is legal inside a name.Excel normalizes three things on entry that Fx deliberately leaves alone, each needing the workbook's sheet names in order: the ordering of the two ends, a degenerate range (
Jan:Jan!A1toJan!A1), and each end's case. SofixFormulaRangesnormalizes the range of a 3-D reference but never its sheet range.Per-end quoting (
foo:'bar'!A1) is not an Excel spelling at all: Excel storesSUM(plain:'has space'!A1)as typed and evaluates it to#NAME?. Fx accepts it anyway, to be forgiving of other producers and redistributes the quoting over the whole prefix on output. Excel does write that spelling, but only for a sheet range that lost an end —Nope:'Mar'!A1where the first end no longer names a sheet, andJan:'[1]Nope'!A1, with a manufactured external link, where the second does not. The bracketed form is refused here, a workbook being nameable only ahead of the whole prefix; the other is read as the sheet rangeNope:Mar, sodocs/Prefixes.mdwarns a caller resolving the sheet slot what it may be looking at.A sheet range is one only in front of a cell reference. Measured in Excel on sheets
Alpha,BetaandGammawith a tableTable1onBeta, the same prefix in front of a defined name or a structured reference is read two other ways, and never as a span:Alpha:Gamma!A1(control)Alpha:Gamma!SomeNameAlphatoGamma!SomeName'Alpha:Gamma'!SomeName[n]!SomeName, with no sheet at allAlpha:Gamma!Table1[Col]Gamma!discarded as any sheet prefix on a table isAlpha:Table1[Col]'Alpha:Gamma'!Table1[Col][n]!Table1[Col]The bare half of that is followed here.
Alpha:Gamma!SomeNameandAlpha:Gamma!Table1[Col]lex as three tokens — a name, the:operator, and a prefixed operand — so they parse to aBinaryExpressionandparseA1Refreturnsundefinedfor them, exactly as forSheet1!A1:Sheet2!B2.spanTakesOperandprobes the range lexers at the operand and is consulted wherever a colon may join two sheet names; the lone-name branch never asks it, soJan!SomeNameis untouched and only a colon raises the question. The serializers have to follow or the round trip breaks: the per-end quoting above holds in front of a cell reference, and in front of a name or a table the colon is quoted as the ordinary banned character it is, sostringifyA1Ref({ context: [ 'Sales:Marketing' ], name: 'foo' })yields'Sales:Marketing'!foo.What settles the bare reading is a rename, since the storage cannot tell the two apart — Excel keeps the text verbatim either way. Renaming the sheet
AlphaleavesAlpha:Gamma!SomeNameuntouched while both controls move: an ordinaryAlpha!SomeNamebecomesBee!SomeName, and a span over a cell becomesBee:Mar!A1. So theAlphain the subject is an ordinary name that a sheet rename has no reason to touch, and not a sheet reference at all.The two quoted spellings are left as they are, and the limitation note is narrowed to them. Excel reads
'Alpha:Gamma'!SomeNameas a workbook file name with no sheet component at all, and there is no slot for that here — the sheet slot is where such a name would land, and it is the wrong one. Giving it a right one is a larger change than this branch.docs/Prefixes.mdand thesplitSheetRangeandparseA1Refdocstrings say which spellings that still covers, so a caller resolving the sheet slot of a quoted prefix knows what it may be holding.Generated inputs
Sheet names and formulas are generated from a grammar of name spellings, prefix forms and cell addresses, then checked against three properties: the formula lexers and the reference lexers agree on the prefix; parse → stringify → parse reaches a fixpoint with both sheet names intact; and the tokens of an input rejoin to exactly that input. Names are drawn from the whole legal character set and spelled as cells, columns, R1C1 addresses, digits and booleans, at each end of a span independently, with the option flags drawn per case.
Failures are shrunk to a minimal input before they are reported, and the seeds fix what runs, so a failure is reproducible from the line reporting it.
lib/sheetRangeProperties.spec.tsruns a few thousand cases with the suite;scripts/propertySweep.tsruns as many as you have patience for and is not part ofnpm run check.Six defects came out of it, each pinned as a test on its shrunk case:
Ærið!A1was one name token, andForudsætninger!A1broke in two. Unrelated to sheet ranges, so it is extracted as Fix non-ASCII characters breaking unquoted sheet names and names #52, which explains it; this branch drops those two lines once that lands. One assertion needs reconciling at that point rather than merging cleanly:Ærið:Ärger!A1is one sheet range here and two endpoints in Fix non-ASCII characters breaking unquoted sheet names and names #52, correct on each branch alone, and the two sit in different parts oftokenize.spec.tsso git reports no conflict. Both are pinned, here and there, with a comment on each saying which survives.v1.0!A1was the cellV1and three stray tokens..is the one character a sheet name may contain that a range is also allowed to end on, so a range lexer can stop part-way through such a name. A sheet range already had a guard for this; a lone sheet name did not, andstartsSheetRangebecamestartsSheetPrefixto cover both..:Dec!A1was a trim operator with nothing on its left..is a legal sheet name, and the trim range operators are read ahead of everything else, solexRangeTrimandlexRefOpnow give way to a sheet prefix as the range lexers do.a:!!A1was a sheet range over a sheet named!. Where the colon is followed straight by!the second name is missing, whichlexContextUnquotednoted and then went on scanning for a later!to close the prefix at.RC:':'!A1was a sheet range over a sheet named:. A separately-quoted endpoint could contain the one character Excel forbids in a sheet name.splitSheetRangethen refused to divide the scope, so the translators, which ask it, did not know to quote the prefix, and the same string was a sheet range in A1 notation and a range operation in R1C1.'False:Jan'!A1was written back asFalse:Jan!A1. BareTRUE!A1lexes as the boolean joined to a reference, so such a name needs quotes as much as a range-like or a digit-leading one does. Per-end quoting is what exposed it: the whole-scope quoting it replaced had kept the pair together by accident, since:is a banned character. The same rule fixes a case that has nothing to do with sheet ranges: onmaster,fixFormulaRanges("=SUM('TRUE'!A1)")returns=SUM(TRUE!A1), dropping quotes a lone boolean-reading sheet name needs. Excel refuses to open a workbook containing the unquoted spelling, so that rewrite turns a safe formula into one that costs the file — worth knowing separately from the sheet-range work, since it reaches any caller that normalizes a formula.One gap is qualified rather than fixed, and the qualification is written into the properties with its reason: a number and a boolean literal are both lexed ahead of a name on the formula path and have no counterpart on the reference path, so bare
12!A1andTRUE!A1are a prefix toparseA1Refand something else totokenize. That predates sheet ranges and applies to a lone sheet name as much as for one end of a span. Neither spelling is one a writer produces — Excel quotes an all-digit name, andneedQuotesnow quotes both — so reaching it takes hand-written input.Drive-by
Both
parseA1Refexamples indocs/Prefixes.mdpassed a{ xlsx: true }option thatparseA1Refdoes not take, so a reader copying either got the context variant with the option silently ignored. They now call the@borgar/fx/xlsxentry point, which is where the xlsx counterparts actually live. One of the two predates 3-D references.Notes
docs/Prefixes.mdcovers the above, andfixTokenRangesandfixFormulaRangesstate in their own docstrings that the sheet range of a 3-D reference is exempt from normalizing, so the exemption reaches the generated docs.docs/API.mdis regenerated rather than hand-edited; the version is left alone.One pre-existing issue was found and left alone:
fixRangesandtranslateToA1disagree about quoting the prefix to the right of a range operator, the latter adding quotes the former strips.