Skip to content

Add opt-in shaped reward for color codeword - #2

Draft
rishabhadiga2001 wants to merge 4 commits into
fix/mcp1-harness-compatfrom
rishabh/color-codeword-shaped-reward
Draft

Add opt-in shaped reward for color codeword#2
rishabhadiga2001 wants to merge 4 commits into
fix/mcp1-harness-compatfrom
rishabh/color-codeword-shaped-reward

Conversation

@rishabhadiga2001

Copy link
Copy Markdown

Summary

  • add an opt-in partial-match reward weight for incorrect color-codeword answers
  • preserve exact-only reward behavior by default and keep exact answers at 1.0
  • add focused strict/default and shaped-reward tests

Test plan

  • PYTHONPATH=deps/verifiers:deps/verifiers/environments/color_codeword_v1 /private/tmp/prime-verifiers-env/bin/python -m pytest deps/verifiers/tests/v1/test_color_codeword_tasks.py -q

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant