Skip to content

Fix #208: draw problem 59's hidden array from a secret per-test seed - #211

Merged
joyemang33 merged 1 commit into
FrontierCS:mainfrom
whuang369:fix/208-problem-59-seed
Nov 1, 2026
Merged

joyemang33 merged 1 commit into
FrontierCS:mainfrom
whuang369:fix/208-problem-59-seed

Conversation

@whuang369

@whuang369 whuang369 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

The interactor built the hidden array with testlib's rnd, which registerInteraction() never seeds, and the .ans files were empty. Every test with the same n got the same array, and a program that replays testlib's generator, as in the issue, scored 100 with zero queries on all 10 tests.

Each testdata/N.ans now holds a distinct secret 63-bit seed, drawn with Python's secrets.randbits(63). The interactor reads it before printing n and builds the array from std::mt19937_64(seed) with the same swap rule as before. mt19937_64 is fully specified by the C++ standard, so a seed gives the same array with any compiler and testlib build; testlib's rnd keeps only 48 bits of state, and its stream depends on the testlib version and build flags. Problem 143 already reads hidden mt19937_64 seeds from its .ans files the same way. A missing seed is a judge failure (_fail), and the seed is never printed to the contestant or the log. Like every hidden answer file, the seeds are public in this repo; the fix removes the replay that needs no outside data.

The arrays still follow the statement, so honest scores barely change: the archived algorithmic/solutions/59/gpt5.2_3.cpp averages 64.064088 on the old arrays and 64.064036 on the new ones.

Verified in the frontier-cs-judge image (g++ 11.4, -O2 -std=gnu++17) and with g++ 15.2: all 25 tests in the new algorithmic/tests/test_problem59.py pass. They check that the replay from the issue is rejected on tests 2 to 10 (test 1 has n = 1, where zero queries is the honest answer), pin each hidden array by hash, check that tests with equal n get different arrays, and check that the seed never appears in the interactor's output. On main, 24 of them fail and the replay scores 100 on every test. test_problem59_budget_reference.py now writes a seed to its answer file, and its 6 tests pass.

Relation to #209: #209 also puts a per-test seed in the .ans files, but seeds testlib's rnd with it instead of using mt19937_64. Both PRs add algorithmic/tests/test_problem59.py, so only one can merge as is.

Fixes #208.

…test seed

The interactor built the hidden array with testlib's rnd, which
registerInteraction() never seeds, and the .ans files were empty. Every
test with the same n got the same array, and a program that replays
testlib's generator, as in the issue, scored 100 with zero queries on
all 10 tests.

Each testdata/N.ans now holds a distinct secret 63-bit seed, drawn with
Python's secrets.randbits(63). The interactor reads it before printing
n and builds the array from std::mt19937_64(seed) with the same swap
rule as before. mt19937_64 is fully specified by the C++ standard, so a
seed gives the same array with any compiler and testlib build; testlib's
rnd keeps only 48 bits of state, and its stream depends on the testlib
version and build flags. Problem 143 already reads hidden mt19937_64
seeds from its .ans files the same way. A missing seed is a judge
failure (_fail), and the seed is never printed to the contestant or the
log. Like every hidden answer file, the seeds are public in this repo;
the fix removes the replay that needs no outside data.

The arrays still follow the statement, so honest scores barely change:
the archived algorithmic/solutions/59/gpt5.2_3.cpp averages 64.064088
on the old arrays and 64.064036 on the new ones.

Tested with g++ 15.2 (-O2 -std=gnu++17): all 25 tests in the new
algorithmic/tests/test_problem59.py pass. They check that the replay
from the issue is rejected on tests 2 to 10 (test 1 has n = 1, where
zero queries is the honest answer), pin each hidden array by hash,
check that tests with equal n get different arrays, and check that the
seed never appears in the interactor's output. On main, 24 of them fail
and the replay scores 100 on every test.
test_problem59_budget_reference.py now writes a seed to its answer
file, and its 6 tests pass.

Fixes FrontierCS#208.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@joyemang33
joyemang33 merged commit 6629bc3 into FrontierCS:main Nov 1, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem 59 interactor is non-adaptive with a fixed RNG seed, so a zero-query program scores 100

2 participants