fix: don't copy the driver environment into the caller's env_vars dict

[David-Wu1119]

What does this PR do ?

Stops RayWorkerGroup from writing the driver's environment into the env_vars dict it is given, which put every environment variable (tokens included) into each checkpoint's config.yaml.

_create_workers_from_bundle_indices merged os.environ into env_vars in place. That dict often belongs to the run config:

• lm_policy.py passes config["dtensor_cfg"].get("env_vars", {}) unchanged (the Megatron path already copies it with dict(...)); lm_value.py and teacher_worker_group.py do the same.
• PolicyConfig / DTensorConfig are TypedDicts stored as-is inside the pydantic MasterConfig, so master_config.policy[...]["env_vars"] is that same dict object (checked with pydantic 2.13.5).
• GRPO saves master_config with each checkpoint (checkpointer.init_tmp_checkpoint(..., master_config) → yaml.safe_dump(run_config.model_dump(mode="json")) in utils/checkpoint.py).

So a run that sets, for example, policy.dtensor_cfg.env_vars: {NCCL_DEBUG: WARN} ends up with the whole driver environment (HF_TOKEN, WANDB_API_KEY, cloud credentials, ...) under that key in every checkpoint's config.yaml.

The change builds a new dict instead, env_vars = {**os.environ, **env_vars}, so caller values still take precedence and workers see exactly what they did before.

Issues

None filed.

Usage

No API change.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• New test test_env_vars_argument_is_not_modified in tests/unit/distributed/test_worker_groups.py (CPU Ray cluster, like its neighbours). On main it fails with the dict holding the full environment; with this change it passes.
• tests/unit/distributed/test_worker_groups.py on this branch: 29 passed, 1 skipped (local CPU Ray, Python 3.13, torch 2.11), including test_environment_variable_precedence_full.
• ruff / ruff-format pass. pyrefly check nemo_rl/distributed/worker_groups.py reports the same 5 findings on main and on this branch. I couldn't run the uv-based hooks as configured, because Python 3.13.14 isn't available for macOS arm64 yet.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.