Repository navigation
Fix #205: drop problem 23's unchecked accept for Tid % 100 == 0 - #210
Merged
joyemang33 merged 1 commit intoNov 1, 2026
Merged
Conversation
…== 0 check_Tid() returned "ok Test 0." for any Tid divisible by 100 before the submitted A=B program ran, so even an empty program scored 100 on such a test. At c7ec558 the only tests were Tid 0, 100 and 200, which made the whole problem free. None of the 16 released test ids on main is a multiple of 100, so current scores are not affected. Tid % 100 picks the test set. dict maps 0 to 20 to themselves and anything else to 0, but work() has only 17 generators, and generator 0 is empty. So 0 got the free accept, 17 to 20 indexed past the end of vf (undefined behavior), and 21 to 99 ran no test strings and printed "Ratio: nan". - Remove the shortcut. - work() fails with _fail (a data error) when Tid % 100 selects no generator. - validate_test() fails with _fail when a test generates no test strings, so no test can pass without running the program. - Remove the Scoring paragraph in statement.txt that documented the shortcut. The released tests (Tid % 100 in 1 to 16) are unchanged: the old and new checkers give byte-identical output on all 16 of them for 8 programs (empty, "a=b", the reference program from the .ans files and 5 archived solutions). Tested with g++ 15.2 (-O2 -std=gnu++17): all 33 tests in the new algorithmic/tests/test_problem23.py pass. On main, the 17 that cover Tid % 100 == 0 and the ids without a test set fail, and the 16 released-test cases pass. Fixes FrontierCS#205. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
check_Tid()returnedok Test 0.for any Tid divisible by 100 before the submitted A=B program ran, so even an empty program scored 100 on such a test. At c7ec558 the only tests were Tid 0, 100 and 200, which made the whole problem free. None of the 16 released test ids on main is a multiple of 100, so current scores are not affected.Tid % 100 picks the test set.
dictmaps 0 to 20 to themselves and anything else to 0, butwork()has only 17 generators, and generator 0 is empty. So 0 got the free accept, 17 to 20 indexed past the end ofvf(undefined behavior), and 21 to 99 ran no test strings and printedRatio: nan.work()fails with_fail(a data error) when Tid % 100 selects no generator.validate_test()fails with_failwhen a test generates no test strings, so no test can pass without running the program.statement.txtthat documented the shortcut.The released tests (Tid % 100 in 1 to 16) are unchanged: the old and new checkers give byte-identical output on all 16 of them for 8 programs (empty,
a=b, the reference program from the .ans files and 5 archived solutions).Verified in the frontier-cs-judge image (g++ 11.4, -O2 -std=gnu++17) and with g++ 15.2: all 33 tests in the new
algorithmic/tests/test_problem23.pypass. On main, the 17 that cover Tid % 100 == 0 and the ids without a test set fail, and the 16 released-test cases pass.Relation to #207: #207 also removes the shortcut, but fills set 0 with sets 1 and 2, so Tid % 100 == 0 and 21 to 99 become real tests instead of data errors. It does not change
statement.txt, which still documents the shortcut, and Tid % 100 in 17 to 20 still indexes pastvf. Both PRs addalgorithmic/tests/test_problem23.py, so only one can merge as is.Fixes #205.