Repository navigation
Conversation
Use the preprocessed image for rollout and restore visual tensors received as JSON lists before collation.
kevssim
left a comment
There was a problem hiding this comment.
🤖 本 review 由 AI 生成,仅供参考,请结合实际情况判断。
collator 用 torch.as_tensor 兼容 HTTP 传来的列表,这部分没有问题,行为与原来一致。encode_prompt 优先取引擎编码后的图片,在 vllm/sglang/lmdeploy 下是正确的,但在 transformers 后端下会取到处理后的张量,建议按后端区分(见行内评论)。
| media = MediaBundle.from_sources(sample, feature) | ||
| # Engine encoding retains the images after the same resize used for | ||
| # the expanded training feature. | ||
| media = MediaBundle.from_sources({"images": engine_feature.get("images")}, sample, feature) |
There was a problem hiding this comment.
🤖 以下为 AI review 意见,供参考。
[P2] backend="transformers" 时,engine_feature["images"] 可能是处理过的张量,不是图片
只有在 vllm、sglang、lmdeploy 模式下,_encode_truncated 才会把预处理后的 inputs.images 写入 encoded["images"](base.py:2347-2357)。backend="transformers" 时,render_mode 返回 "transformers",走的是模型自己的 _encode。这时 glm(glm.py:615)、internlm(internlm.py:112)、llava(llava.py:386)、yi(yi.py:47)、deepseek(deepseek.py:383,单个 [(crop, ori)])等模板写入的都是处理后的张量。twinkle 的 generation.py:102 和 transformers_sampler.py:298 都会以 backend="transformers" 调用 encode_prompt。结果是:RolloutState.media.images 变成张量;twinkle 的 _media_ref 遇到张量会抛 TypeError;deepseek 多图时 _check_engine_placeholders 只看到 1 张图而报错。
建议只在引擎模式下使用 engine_feature 里的图片:
| media = MediaBundle.from_sources({"images": engine_feature.get("images")}, sample, feature) | |
| engine_images = engine_feature.get("images") if backend in ("vllm", "sglang", "lmdeploy") else None | |
| media = MediaBundle.from_sources({"images": engine_images}, sample, feature) |
a0bdb25 to
a1df7ca
Compare
a1df7ca to
8ad3623
Compare
Use the preprocessed image for rollout and restore visual tensors received as JSON lists before collation.
PR type
PR information
背景
在 client-server 模式下运行图文 GRPO 时,Swift 训练模板会按
max_pixels缩放图片,采样请求却可能仍携带原图,导致 vLLM 生成回答时使用的视觉输入与训练侧不一致。另外,视觉张量经 HTTP JSON 传输后会成为嵌套列表,服务端直接拼批会报错。改动
encode_prompt()优先使用引擎编码阶段保留的预处理图片,让采样和训练基于同一张缩放后的图。torch.as_tensor()转为张量,兼容 HTTP 列表和本地 Tensor。_data_collator_mm_data()对pixel_values、image_sizes和pixel_values_videos逐项转为张量后再拼批。Checklist