closure: keep the lambda compiler's recursion inside the C stack - #140
Open
buzzom wants to merge 1 commit into
Open
closure: keep the lambda compiler's recursion inside the C stack#140buzzom wants to merge 1 commit into
buzzom wants to merge 1 commit into
Conversation
compile_value() guards its recursion with a counter starting at
MAX_LAMBDA_LEVELS = 0x8000 (32768), but the C stack cannot take that
many levels: one level of lambda nesting costs about 430 bytes of stack
(compile_value() plus compile_efun_call(), gcc -O2 on x86_64), so with
the common 8 MB stack limit the driver dies at a nesting depth of about
19500 - far below the limit that is supposed to protect it. A deeply
nested value passed to lambda() crashes the driver instead of raising
the "Too deep recursion inside lambda()" error:
mixed v = 1;
for (int i = 0; i < 25000; i++) v = ({ #'+, 1, v });
lambda(0, v); /* segfault */
Measured on master: a nesting depth of 19000 compiles, 20000 crashes.
An AddressSanitizer build needs about 1.1 KB per level and already dies
between 7000 and 8000.
compile_lvalue() has the same recursion but no guard at all - it never
touches the counter, so lowering the limit alone does not help this
path:
mixed lv = 'a;
for (int i = 0; i < 100000; i++) lv = ({ #'[, lv, 0 });
lambda(({'a}), ({ #'=, lv, 1 })); /* segfault */
Lower MAX_LAMBDA_LEVELS to 1000 and let compile_lvalue() take part in
the depth accounting through a counting wrapper, following the
regmatch()/regmatch_int() pattern in regexp.c. 1000 levels stay below
1 MB of C stack in a normal build, survive a 2 MB stack limit even in
a sanitizer build, and match MAX_JSON_NESTING_DEPTH in pkg-json.c.
With the new limit a value nesting depth of 998 still compiles and 999
raises the error cleanly.
Also drop the stray semicolon at the end of the #define; it was
harmless only because the macro is used solely in statement position.
Adds four cases to t-efuns.c: values and lvalues nested well below the
limit still compile (and the value case evaluates correctly), and both
kinds of 2000-deep nesting raise a catchable error. Without the fix
both error tests fail with "There was no error." The full test suite
and an ASan/UBSan run pass with the change.
buzzom
force-pushed
the
lambda-recursion-depth
branch
from
July 31, 2026 15:50
6931395 to
5e8ba33
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
compile_value()guards its recursion with a counter starting atMAX_LAMBDA_LEVELS = 0x8000(32768), but the C stack cannot take that manylevels, so the guard never fires: the driver crashes first. A deeply nested
value passed to
lambda()segfaults instead of raising the existing"Too deep recursion inside lambda()" error:
compile_lvalue()recurses into nested lvalues without going throughcompile_value()and never touches the counter at all, so it is a second,completely unguarded path that a lower limit alone would not fix:
Measurements
All numbers from current master, gcc on x86_64 Linux, 8 MB stack limit
(
ulimit -s 8192), bisected with the reproduction above:-O2compile_value()+compile_efun_call())-O1ASan/UBSanThe lvalue chain is cheaper per level (25000 levels survive, 100000 crash at
-O2), but unbounded all the same.Fix
MAX_LAMBDA_LEVELSto 1000. That stays below 1 MB of C stack in anormal build, survives even a 2 MB stack limit in a sanitizer build
(verified with
ulimit -s 2048), and matchesMAX_JSON_NESTING_DEPTHinpkg-json.c(pkg-json: bound the recursion in json_serialize() #136). With the new limit a value nesting depth of 998 stillcompiles and 999 raises the error cleanly.
compile_lvalue()take part in the depth accounting through a countingwrapper around the renamed
compile_lvalue_int(), following theregmatch()/regmatch_int()pattern inregexp.c. The function has morethan ten
returnstatements, so a wrapper is less error-prone thanrestoring the counter at every exit.
#define; it was harmless onlybecause the macro is used solely in statement position.
Alternatives considered
stack address at
lambda()entry and compare against a budget derived fromgetrlimit(RLIMIT_STACK). It would adapt to the actual frame size, but thedriver currently calls
getrlimit()nowhere, so this adds new portabilitysurface for a limit nobody should ever reach legitimately. I am happy to
follow up with this variant if you prefer it.
assert_stack_gap()only guards the gap between heap and C stack; on a64-bit host with the stack far above the heap it never fires (same
reasoning as in pkg-json: bound the recursion in json_serialize() #136), and it would not have caught either crash.
both builds (4096 × 1.1 KB ≈ 4.5 MB), but leaves no margin under smaller
stack limits: with
ulimit -s 2048a sanitizer build dies far below 4096levels. Since even 1000 levels of nesting is far beyond any hand-written or
generated lambda expression I could find, I chose the safer value.
LD_REGEXP_RECURSION_LIMIT.Possible, but the safe value depends on build flags rather than on the
mudlib, so a compile-time constant with a documented rationale seemed more
honest.
Tests
Four new cases in
t-efuns.c:Without the fix, both error tests fail with "There was no error." (and depths
of 25000 value levels / 100000 lvalue levels segfault the driver). The full
test suite passes in a normal build and in an ASan/UBSan build; the
sanitizers report nothing from
closure.c.The positive value test nests to the left (
({ #'+, val, 1 })) on purpose:compile-time recursion depth is the same in both directions, but a
right-nested chain would also grow the evaluator stack at call time, which is
a separate resource with its own limit.
Related, not part of this PR
compile_value()restores the counter on normal exit but not whenlambda_error()unwinds. That is pre-existing and harmless: the counter isreinitialised at every
lambda()call.