Repository navigation
refactor(workers)!: unify worker class selection through config - #4176
yuchenwang3 wants to merge 3 commits into
Conversation
ff7adc1 to
fe6d468
Compare
|
Rebased onto main ( The focused CPU selection passes 84 tests, including the FP32 LM-head cases. Changed-file formatting, lint, TOML, and recipe checks pass. Targeted Pyrefly reports zero errors under the repository configuration. The full-repository Pyrefly hook could not start because the local Automodel submodule is not populated; GPU/distributed validation still requires the full environment and maintainer-authorized CI. |
fe6d468 to
ea794c2
Compare
|
Rebased onto main ( The focused CPU selection passes 88 tests (39 deselected), including the FP32 LM-head cases. Changed-file formatting, lint, TOML, and recipe checks pass; targeted Pyrefly reports zero shown errors under the repository configuration. Full-repository Pyrefly remains blocked locally by the unpopulated Automodel submodule. GPU/distributed validation was not run and still requires maintainer-authorized CI for this SHA. The PR description now contains the current results and reproduction command. |
ea794c2 to
82b433e
Compare
82b433e to
a297e5d
Compare
|
@yuki-97 Rebased onto main at The focused CPU selections pass 79 + 21 tests; changed-file lint/format and targeted type checks also pass. GPU/distributed validation has not run. Could you review the worker-selection path for #4032 and help authorize CI for this SHA? |
a297e5d to
24c2760
Compare
|
@yuki-97 Rebased on main and fixed the conflicts. The template and worker tests now use the Automodel names. All 144 focused CPU tests pass; GPU tests still need CI. Could you review the worker-selection change and authorize CI for |
Resolve ModelOpt requirements through worker_extension_cls_fqn, deprecate the Policy constructor override, and migrate the template and teacher configuration. BREAKING CHANGE: custom quantized worker subclasses passed through the Policy constructor must now match the backend ModelOpt worker, as required for config overrides. Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com>
24c2760 to
8ee8c68
Compare
There was a problem hiding this comment.
hi @yuchenwang3 , sorry for late reply. thanks for helping refactor this, much more clean than before!
@RayenTian could you help review it?
Preserve quantization and worker-extension cleanup assertions alongside the upstream teacher MTP isolation matrix. Validation: 231 focused CPU tests passed, 1 expected skip. Ruff lint, import ordering and formatting passed for all 15 PR Python files. Assisted-by: OpenAI Codex Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com>
RayenTian
left a comment
There was a problem hiding this comment.
Generated by OpenAI Codex
| from nemo_rl.models.policy import PolicyConfig | ||
|
|
||
|
|
||
| def resolve_worker_cls( |
There was a problem hiding this comment.
C2 · 1 action item.
TL;DR — Reuse the existing configuration utilities for the shared worker-selection helper.
AI-1
Move this helper into nemo_rl/utils/config.py. Keep resolve_policy_worker_cls and resolve_generation_worker_cls, with their respective backend maps, in their current modules. Update the policy import and generation import, then delete this module and its pyrefly entry. Preserve the TYPE_CHECKING-only model imports in the destination. This is a multi-file move, so it cannot be represented by one in-place suggestion.
|
|
||
| `policy.worker_extension_cls_fqn` and | ||
| `policy.generation.worker_extension_cls_fqn` select the policy and vLLM workers. | ||
| When `quant_cfg` is set, the FQN must match the ModelOpt worker for the selected |
There was a problem hiding this comment.
C1 · 1 action item.
TL;DR — The base recipe still says worker extensions and quantization are mutually exclusive.
The new guide accepts an exact ModelOpt worker FQN, making the base recipe's “mutually exclusive with quant_cfg” guidance stale in this PR. This is a non-blocking documentation issue.
AI-1
Update the policy comment and generation comment to say that quantization requires the backend's exact ModelOpt worker and that null auto-selects it. These lines are outside this PR's diff, so this cannot be an in-place suggestion.
Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com>
|
@RayenTian Both comments are addressed in the current head 079f0f7: the helper moved into |
What does this PR do?
Make
worker_extension_cls_fqnthe common selection path for policy and vLLM workers. Today a config-provided FQN is rejected wheneverquant_cfgis set, even if it names the correct quantized worker, while the constructor override can replace that worker without the same check.The shared resolver fills an omitted FQN with the backend's quantized worker and warns, accepts an exact match, and rejects a mismatch with the expected FQN before allocating workers. Existing quant-only recipes keep automatic worker selection.
The
Policyconstructor argument now emits a deprecation warning and feeds the config path. The research template selects its extension in YAML, and frozen teachers clear the copied student extension alongside quantization settings.Issues
Closes #4032.
Usage and compatibility
Leave
worker_extension_cls_fqnunset to keep automatic selection, or set it to the FQN printed by the warning. An explicit quant worker must match the policy backend or vLLM sync/async mode. The Automodel backend usesautomodel_cfg. The CLI entrypoints' existingcheck_outdated_config()rejects the olddtensor_cfgblock and the removed_v2key for either value, including when an extension is provided.Compatibility change: custom subclasses passed through
Policy(worker_extension_cls_fqn=...)withquant_cfgare now rejected, just like config-provided mismatches. Non-quantized custom extensions remain supported. The deprecated argument still works when it satisfies the config validation rules; conflicting constructor/config values remain errors.Validation (October 8)
Rebased onto main
2c2ecd2ea814d666edae2a8ce8b458403cb692b1at24c27600e4df9330b48e90fb78850ae2e0f7150e. The conflicts retain upstream vLLM FP32 patch imports and the Automodel class/config rename. The template YAML and tests use the new FQNs. Stale-config rejection stays in main's unchangedcheck_outdated_config()entrypoint path; eight cases cover quantization with an omitted or explicit worker FQN, the old/new backend block names, and both_v2values.HF_HOMEwas unset; the same failures reproduce on main, and all 18 pass after setting it. No test-code workaround or project dependency change was made.git diff --checkpassed. Targeted Pyrefly on the shared helper and policy/vLLM resolvers reported zero shown errors (9 ignored under repository configuration). The full-repository Pyrefly hook was skipped..venv-4032,--no-sync, and--noconftest. Full-repository type checking, distributed tests, and GPU model execution were not run. These tests validate worker selection; upstream's Automodel quantized worker is still a placeholder. NVIDIA-runner CI requires maintainer authorization for this SHA.Reproduce the focused selections in a configured test environment:
--noconftestskips the repository's autouse Ray/GPU-monitor fixtures for these CPU cases. Locally, these commands usedUV_PROJECT_ENVIRONMENT=.venv-4032 uv run --no-sync.Pre-checks
Assisted-by: OpenAI Codex