Repository navigation
Add Qwen3-VL loading and tool-use examples - #493
Meihan-chen wants to merge 7 commits into
Conversation
main can already export Qwen3-VL to HF: megatron_to_hf/qwen3_vl.py and its dispatch entry are there. The loading direction is missing, so a Qwen3-VL checkpoint cannot be brought in to begin with. Add the loader, register it, and add the native model. The loader reads the language tower through a thin view that rewrites model.* to model.language_model.*, leaves model.visual.* alone, and resolves a tied output layer back to the language embedding, so the existing qwen loader is reused rather than duplicated. Also fix vocab-padding removal on export. remove_padding() matched embedding.word_embeddings.weight and output_layer.weight exactly, which a Qwen3-VL parameter never is: its names carry a language_model. prefix, so padded rows were exported as real vocabulary. The prefix is stripped before the comparison, which is a no-op for every other model. test_loader_scope_stays_explicit pins the registered loader set on purpose, so it is updated here rather than relaxed. Verified: tests/test_hf_to_megatron.py and tests/test_qwen3_vl_native.py total 85 passed, including a language-tower round trip, the tied-output case, and padding removal checked for both prefixed and unprefixed names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Meihan-chen <zr010426ztt@outlook.com>
…g it connect_rollout_engines() assigned self.rollout_engines to the incoming engines and then called disconnect_rollout_engines(), which reads that same attribute. On a reconnect the teardown therefore went to the new engines, which had never joined a group, while the group being replaced stayed connected and its trainer kept its peers. Swap the two statements. The first connect is unaffected: __init__ sets _trainer to None, so disconnect short-circuits before reading anything. test_connect_replaces_existing_trainer asserted the old behavior, by checking the teardown reached the incoming engine. It now gives the updater a previous engine and asserts the teardown reaches that one and not the new one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Meihan-chen <zr010426ztt@outlook.com>
Three fixes in the deepscaler rule-based reward:
- "\boxed" written as a Python literal is a backspace character, so
responses can arrive with the marker already destroyed. Restore it
before parsing rather than scoring the answer as absent.
- With enable_thinking=false the turn ends at <|im_end|> and there is
no </think>. The span up to it is graded, so an earlier boxed answer
is not dropped because an explanation follows.
- split("###Response") is bounded to one split, so a response
containing the marker twice no longer changes which part is graded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Meihan-chen <zr010426ztt@outlook.com>
…r ids Two example bodies with their data preparation, serving helpers and run scripts, plus the rollout and reward code they need. examples/multi_agent threads a worker id through generate_response() and records it, with the role and the parent sample's group and index, in each generated sample's metadata, so a multi-agent rollout can be traced back to the agent and the parent that produced it. Examples only; nothing under vime/ changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Meihan-chen <zr010426ztt@outlook.com>
pytest.importorskip("megatron.core") is satisfied by the stub an earlier
test installs, so in a whole-suite run these eleven tests reached the
import and errored on a missing tensor_parallel instead of skipping.
Guard on megatron.core.tensor_parallel, which is what the native model
imports.
Also drop a VIME_PLATFORM setenv carried over from a platform layer that
does not exist here, and which nothing reads.
CI runs test files individually, so this changes nothing there: the file
is 11 passed either way. In a single-process run of the whole suite it is
now 11 skipped rather than 11 errors.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Meihan-chen <zr010426ztt@outlook.com>
There was a problem hiding this comment.
Code Review
This pull request introduces several new features and examples, including a multi-agent system with rollout ID sharing, a tool-enabled language model generation framework (Retool) with a safe Python sandbox, a Search-R1 lite implementation supporting local and Google search backends, and native Qwen3-VL model support. Key feedback includes addressing a critical security vulnerability in the Retool sandbox by replacing fragile regex-based safety checks with AST parsing, fixing JSON parsing errors caused by newline replacements, correcting nested dictionary retrieval in the local search server to prevent discarding search results, and resolving a potential runtime AttributeError in the Qwen3-VL model by using the configuration's parameter data type instead of accessing a module's non-existent dtype attribute.
| def _check_code_safety(self, code: str) -> tuple[bool, str]: | ||
| """Check code safety by scanning for dangerous patterns""" | ||
| # Check for dangerous operations | ||
| dangerous_patterns = [ | ||
| r"import\s+os", | ||
| r"import\s+sys", | ||
| r"import\s+subprocess", | ||
| r"import\s+shutil", | ||
| r"import\s+glob", | ||
| r"import\s+pathlib", | ||
| r"__import__", | ||
| r"eval\s*\(", | ||
| r"exec\s*\(", | ||
| r"open\s*\(", | ||
| r"file\s*\(", | ||
| r"input\s*\(", | ||
| r"raw_input\s*\(", | ||
| r"compile\s*\(", | ||
| r"execfile\s*\(", | ||
| r"getattr\s*\(", | ||
| r"setattr\s*\(", | ||
| r"delattr\s*\(", | ||
| r"hasattr\s*\(", | ||
| r"globals\s*\(", | ||
| r"locals\s*\(", | ||
| r"vars\s*\(", | ||
| r"dir\s*\(", | ||
| r"type\s*\(", | ||
| r"isinstance\s*\(", | ||
| r"issubclass\s*\(", | ||
| r"super\s*\(", | ||
| r"property\s*\(", | ||
| r"staticmethod\s*\(", | ||
| r"classmethod\s*\(", | ||
| r"__\w+__", # double underscore methods | ||
| ] | ||
|
|
||
| for pattern in dangerous_patterns: | ||
| if re.search(pattern, code, re.IGNORECASE): | ||
| return False, f"Code contains dangerous pattern: {pattern}" | ||
|
|
||
| # Check imported modules | ||
| import_pattern = r"import\s+(\w+)" | ||
| from_pattern = r"from\s+(\w+)" | ||
|
|
||
| imports = re.findall(import_pattern, code) | ||
| froms = re.findall(from_pattern, code) | ||
|
|
||
| all_imports = set(imports + froms) | ||
| for imp in all_imports: | ||
| if imp not in self.allowed_modules: | ||
| return False, f"Import of '{imp}' is not allowed" | ||
|
|
||
| return True, "Code is safe" |
There was a problem hiding this comment.
The _check_code_safety method uses regular expressions to detect dangerous imports and operations. However, regex-based parsing of Python code is extremely fragile and easy to bypass. For example, import math, os bypasses both the dangerous_patterns check (which expects import\s+os) and the import_pattern check (which only extracts the first module math due to the \s+(\w+) pattern). This allows an attacker to import and execute arbitrary code (e.g., os.system) on the host system, leading to a complete sandbox escape.\n\nUsing Python's built-in ast (Abstract Syntax Tree) module is the standard, robust, and secure way to parse and inspect Python code structure safely.
def _check_code_safety(self, code: str) -> tuple[bool, str]:\n \"\"\"Check code safety by parsing the AST to prevent sandbox escape.\"\"\"\n import ast\n\n try:\n tree = ast.parse(code)\n except SyntaxError as e:\n return False, f\"Syntax error: {e}\"\n\n for node in ast.walk(tree):\n # Check imports (e.g., import os, or import math, os)\n if isinstance(node, ast.Import):\n for alias in node.names:\n base_module = alias.name.split(\".\")[0]\n if base_module not in self.allowed_modules:\n return False, f\"Import of '{alias.name}' is not allowed\"\n elif isinstance(node, ast.ImportFrom):\n if node.module:\n base_module = node.module.split(\".\")[0]\n if base_module not in self.allowed_modules:\n return False, f\"Import from '{node.module}' is not allowed\"\n\n # Check for dangerous builtins\n if isinstance(node, ast.Name):\n if node.id in {\n \"eval\",\n \"exec\",\n \"open\",\n \"input\",\n \"raw_input\",\n \"compile\",\n \"globals\",\n \"locals\",\n \"__import__\",\n \"getattr\",\n \"setattr\",\n \"delattr\",\n \"hasattr\",\n }:\n return False, f\"Use of builtin '{node.id}' is not allowed\"\n\n # Check for dunder attributes/methods access\n if isinstance(node, ast.Attribute):\n if node.attr.startswith(\"__\") and node.attr.endswith(\"__\"):\n return False, f\"Access to double underscore attribute '{node.attr}' is not allowed\"\n\n return True, \"Code is safe\"| # Replace newlines in string values with \n | ||
| json_str = json_str.replace("\n", "\\n") | ||
| tool_call_data = json.loads(json_str) |
There was a problem hiding this comment.
The code attempts to clean up the JSON string by replacing all newlines with \\n (json_str = json_str.replace(\"\\n\", \"\\\\n\")). However, this replaces newlines outside of JSON string literals (e.g., between keys or after braces in pretty-printed JSON), turning valid JSON into completely invalid JSON (e.g., {\n becomes {\\n which is a syntax error). This causes json.loads to always fail with a JSONDecodeError whenever the model outputs formatted or pretty-printed JSON.\n\nUsing json.loads(json_str, strict=False) is the standard and robust way to allow unescaped control characters (like raw newlines) inside JSON string values without breaking the overall JSON structure.
| # Replace newlines in string values with \n | |
| json_str = json_str.replace("\n", "\\n") | |
| tool_call_data = json.loads(json_str) | |
| tool_call_data = json.loads(json_str, strict=False) |
| if isinstance(item, dict): | ||
| # Access the document dict first, then get contents | ||
| content = item.get("contents", "") | ||
|
|
||
| if content: | ||
| # The contents are already in the correct format: '"Title"\nText content...' | ||
| # Just pass through as-is to match google_search format | ||
| contexts.append({"document": {"contents": content}}) | ||
| else: | ||
| # Empty content case - provide default values | ||
| contexts.append({"document": {"contents": '"No title."\nNo snippet available.'}}) |
There was a problem hiding this comment.
The code attempts to extract search contents using content = item.get(\"contents\", \"\"). However, the comment on line 83 states that the retrieval server returns a nested structure: {\"document\": {\"id\": \"...\", \"contents\": \"...\"}}. If the structure is nested, item.get(\"contents\") will return None (or \"\"), causing the code to fall back to the else block and append \"No title.\"\\nNo snippet available. for every single search result. This effectively discards all retrieved search results.\n\nWe should safely handle both flat and nested structures by checking if \"document\" is in item.
if isinstance(item, dict):\n # Access the document dict first if nested, then get contents\n document = item.get(\"document\", item) if \"document\" in item else item\n content = document.get(\"contents\", \"\") if isinstance(document, dict) else \"\"\n\n if content:| continue | ||
| if grids is None: | ||
| raise ValueError("Qwen3-VL pixel values require matching grid_thw") | ||
| output = self.model.visual(values.to(dtype=self.model.visual.dtype), grid_thw=grids) |
There was a problem hiding this comment.
The code attempts to cast values to self.model.visual.dtype. However, standard PyTorch nn.Module objects (such as Qwen3VLVisionModel) do not have a .dtype attribute by default, which can raise an AttributeError at runtime. Since self.config.params_dtype is already initialized and used elsewhere in the file, it is much safer and more robust to use self.config.params_dtype directly.
| output = self.model.visual(values.to(dtype=self.model.visual.dtype), grid_thw=grids) | |
| output = self.model.visual(values.to(dtype=self.config.params_dtype), grid_thw=grids) |
Signed-off-by: Meihan-chen <zr010426ztt@outlook.com>
Signed-off-by: Meihan-chen <zr010426ztt@outlook.com>
Add the missing HF-to-Megatron loading path and native training model for Qwen3-VL, plus ReTool and Search-R1 rollout examples. Also fix weight-transfer engine-group reconnection so teardown targets the previous group.
Changes
language_model.prefix.self.rollout_engines. Previously, reconnecting sent teardown to the incoming engines and left the previous group connected. Update the regression test to check which engines receive teardown.Validation
The following tests pass against the proposed code in an isolated Python 3.12 CPU environment:
90 passed, 11 skipped. The 11 native Qwen3-VL tests require
megatron.core.tensor_parallel, which is absent from that environment. Passing tests cover weight mapping/round trips, tied embeddings, vocabulary padding, and engine-group replacement.git diff --checkalso passes.Current limits