Repository navigation
Fix #208: draw problem 59's hidden array from a secret per-test seed - #211
Merged
Merged
Conversation
…test seed The interactor built the hidden array with testlib's rnd, which registerInteraction() never seeds, and the .ans files were empty. Every test with the same n got the same array, and a program that replays testlib's generator, as in the issue, scored 100 with zero queries on all 10 tests. Each testdata/N.ans now holds a distinct secret 63-bit seed, drawn with Python's secrets.randbits(63). The interactor reads it before printing n and builds the array from std::mt19937_64(seed) with the same swap rule as before. mt19937_64 is fully specified by the C++ standard, so a seed gives the same array with any compiler and testlib build; testlib's rnd keeps only 48 bits of state, and its stream depends on the testlib version and build flags. Problem 143 already reads hidden mt19937_64 seeds from its .ans files the same way. A missing seed is a judge failure (_fail), and the seed is never printed to the contestant or the log. Like every hidden answer file, the seeds are public in this repo; the fix removes the replay that needs no outside data. The arrays still follow the statement, so honest scores barely change: the archived algorithmic/solutions/59/gpt5.2_3.cpp averages 64.064088 on the old arrays and 64.064036 on the new ones. Tested with g++ 15.2 (-O2 -std=gnu++17): all 25 tests in the new algorithmic/tests/test_problem59.py pass. They check that the replay from the issue is rejected on tests 2 to 10 (test 1 has n = 1, where zero queries is the honest answer), pin each hidden array by hash, check that tests with equal n get different arrays, and check that the seed never appears in the interactor's output. On main, 24 of them fail and the replay scores 100 on every test. test_problem59_budget_reference.py now writes a seed to its answer file, and its 6 tests pass. Fixes FrontierCS#208. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The interactor built the hidden array with testlib's
rnd, whichregisterInteraction()never seeds, and the .ans files were empty. Every test with the same n got the same array, and a program that replays testlib's generator, as in the issue, scored 100 with zero queries on all 10 tests.Each
testdata/N.ansnow holds a distinct secret 63-bit seed, drawn with Python'ssecrets.randbits(63). The interactor reads it before printing n and builds the array fromstd::mt19937_64(seed)with the same swap rule as before.mt19937_64is fully specified by the C++ standard, so a seed gives the same array with any compiler and testlib build; testlib'srndkeeps only 48 bits of state, and its stream depends on the testlib version and build flags. Problem 143 already reads hiddenmt19937_64seeds from its .ans files the same way. A missing seed is a judge failure (_fail), and the seed is never printed to the contestant or the log. Like every hidden answer file, the seeds are public in this repo; the fix removes the replay that needs no outside data.The arrays still follow the statement, so honest scores barely change: the archived
algorithmic/solutions/59/gpt5.2_3.cppaverages 64.064088 on the old arrays and 64.064036 on the new ones.Verified in the frontier-cs-judge image (g++ 11.4, -O2 -std=gnu++17) and with g++ 15.2: all 25 tests in the new
algorithmic/tests/test_problem59.pypass. They check that the replay from the issue is rejected on tests 2 to 10 (test 1 has n = 1, where zero queries is the honest answer), pin each hidden array by hash, check that tests with equal n get different arrays, and check that the seed never appears in the interactor's output. On main, 24 of them fail and the replay scores 100 on every test.test_problem59_budget_reference.pynow writes a seed to its answer file, and its 6 tests pass.Relation to #209: #209 also puts a per-test seed in the .ans files, but seeds testlib's
rndwith it instead of usingmt19937_64. Both PRs addalgorithmic/tests/test_problem59.py, so only one can merge as is.Fixes #208.