Skip to content

weekly sync: update vime through slime #2185 - #343

Merged
CalvinXKY merged 14 commits into
vllm-project:mainfrom
aoshen02:codex/slime-2185-vime-review
Jul 14, 2026
Merged

CalvinXKY merged 14 commits into
vllm-project:mainfrom
aoshen02:codex/slime-2185-vime-review

Conversation

@aoshen02

@aoshen02 aoshen02 commented Jul 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • mechanically sync slime changes through 680824dd (slime PR #2185)
  • adapt the mechanical result for vime's native vLLM backend and repository-specific CI/docs
  • supersede reverted PR sync: update vime through slime #2185 #338 after explicit human review

Review structure

  • Mechanical reference branch: aoshen02:codex/slime-2185-mechanical-review (0317536c)
  • Merge candidate branch: aoshen02:codex/slime-2185-vime-review (4026f440)
  • Review the mechanical import against main, then review adaptation-only changes between the two branches.

Validation

CI

  • Buildkite #609: passed on 4026f440
  • 26/26 executed jobs passed
  • Megatron GPU suite: 20/20 passed
  • One initial test_qwen3_4B_ppo.py attempt hit a node-local CUDA error 999 during NCCL all_reduce; an unchanged retry on another H100 node passed. The failing GPU still had 49.92 GiB free, so this was not OOM or a deterministic code failure.

100-step convergence run

  • W&B: 5pjx92rf
  • Run name: qwen3-30b-a3b-base-single-node-scaled32p-n16-r20k-pack64k-100step-20260713_093256Z
  • Code tested: 02df2fe78bac9667a3d1eab824efe78b3c26f53a; later PR commits only adjust CI/runtime environment handling and are covered by Buildkite #609.
  • Hardware: one node, 8 × H200 (139.8 GiB visible memory per GPU)
  • Duration: approximately 6 h 31 min
  • Completed rollouts 0..99; the final checkpoint is iteration 99 because rollout IDs are zero-based.
  • Final AIME evaluation at rollout 99:
    • pass@1: 21.25%
    • pass@8: 41.95%
    • pass@16: 45.82%
    • pass@32: 50.00%
    • truncated ratio: 16.77%
image

Reproduction environment

Item Pin
Model Qwen/Qwen3-30B-A3B-Base
Vime commit used by the long run 02df2fe78bac9667a3d1eab824efe78b3c26f53a
Exact original image on h200-1 vllm/vime:pr343-02df2fe-cu13-h2001-backup (sha256:7c6a...); the unqualified tag now points to a later rebuild
Container image ID sha256:7c6a18bae213d50206ec3c917d2e4b4cb2e57d012f240d078d507afefad5d6d1
Base image digest vllm/vllm-openai@sha256:bfdefe75b5c3fb83f4f0fcaae8f39fac87941cbadb05cd2203f44a1689236c71
Runtime Ubuntu 24.04, Python 3.12
PyTorch / CUDA / NCCL 2.11.0+cu130 / 13.0 / 2.28.9
vLLM / Transformers 0.24.0 / 5.12.1
Ray / W&B 2.56.0 / 0.28.0
Weight sync full checkpoint sync
NVLS NCCL_NVLS_ENABLE=0; the test H200 fabric does not expose NVLink multicast

Image build

docker build --progress=plain -f docker/Dockerfile \
  --build-arg BASE_IMAGE=vllm/vllm-openai@sha256:bfdefe75b5c3fb83f4f0fcaae8f39fac87941cbadb05cd2203f44a1689236c71 \
  --build-arg ENABLE_CUDA_13=1 \
  --build-arg INSTALL_FLASHQLA=1 \
  --build-arg VIME_COMMIT=02df2fe78bac9667a3d1eab824efe78b3c26f53a \
  -t vllm/vime:pr343-02df2fe-cu13 .

Dataset preparation

The run used the official ByteDance DAPO/AIME parquet data. Rows are deduplicated by prompt and reward metadata; prompts with conflicting ground-truth rewards are removed.

Artifact Rows SHA-256
dapo-math-17k.parquet 1,791,700 raw 534375d6bb8630d22ab46a56e11f2ffec1d288d8f7d04099bc82d68948705941
dapo-math-17k-cleaned-vime.jsonl 17,391 b912727ac69dea9f1edcdd6c06e6ddf85dc2532a3d7d9e73e5b54f37e74c77e7
aime-2024.parquet 960 raw 12154e38a716d12db5731f9a022ae69a610c4f7d0e0dcc04e902887a686877e7
aime-2024-cleaned-vime.jsonl 30 488aba4c72a0c874012f94d28eb4ba835a3ab2dc848c41d06e06939aa831709a
Dataset conversion script
#!/usr/bin/env python3

import argparse
import json
from pathlib import Path

import pandas as pd


def json_default(value):
    if hasattr(value, "tolist"):
        return value.tolist()
    if hasattr(value, "item"):
        return value.item()
    raise TypeError(f"Unsupported JSON value: {type(value).__name__}")


def canonical_json(value) -> str:
    return json.dumps(
        value,
        ensure_ascii=False,
        sort_keys=True,
        separators=(",", ":"),
        default=json_default,
    )


def prepare(input_path: Path, output_path: Path) -> dict[str, int]:
    frame = pd.read_parquet(input_path)
    frame["_prompt_key"] = frame["prompt"].map(canonical_json)
    frame["_reward_key"] = frame["reward_model"].map(canonical_json)

    deduplicated = frame.drop_duplicates(
        subset=["data_source", "_prompt_key", "ability", "_reward_key"]
    ).copy()
    deduplicated["_reward_count"] = deduplicated.groupby("_prompt_key")[
        "_reward_key"
    ].transform("nunique")
    cleaned = deduplicated[deduplicated["_reward_count"] == 1]

    output_path.parent.mkdir(parents=True, exist_ok=True)
    with output_path.open("w", encoding="utf-8") as output:
        for row in cleaned.itertuples(index=False):
            record = {
                "prompt": row.prompt.tolist() if hasattr(row.prompt, "tolist") else row.prompt,
                "label": row.reward_model["ground_truth"],
                "metadata": {
                    "data_source": row.data_source,
                    "ability": row.ability,
                    "reward_model": row.reward_model,
                    "extra_info": row.extra_info,
                },
            }
            output.write(json.dumps(record, ensure_ascii=False, default=json_default) + "\n")

    return {
        "raw_rows": len(frame),
        "deduplicated_rows": len(deduplicated),
        "cleaned_rows": len(cleaned),
        "ambiguous_rows_removed": len(deduplicated) - len(cleaned),
    }


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("input", type=Path)
    parser.add_argument("output", type=Path)
    args = parser.parse_args()
    stats = prepare(args.input, args.output)
    print(json.dumps(stats, sort_keys=True))


if __name__ == "__main__":
    main()
python prepare_dapo_vime_jsonl.py \
  /root/datasets/byted-dapo-official/dapo-math-17k.parquet \
  /root/datasets/byted-dapo-official/dapo-math-17k-cleaned-vime.jsonl
python prepare_dapo_vime_jsonl.py \
  /root/datasets/byted-dapo-official/aime-2024.parquet \
  /root/datasets/byted-dapo-official/aime-2024-cleaned-vime.jsonl

Model conversion

hf download Qwen/Qwen3-30B-A3B-Base \
  --local-dir /root/models/Qwen3-30B-A3B-Base

source scripts/models/qwen3-30B-A3B.sh
export PYTHONPATH="$PWD:/root/Megatron-LM:${PYTHONPATH:-}"
torchrun --nproc-per-node 8 tools/convert_hf_to_torch_dist.py \
  "${MODEL_ARGS[@]}" \
  --hf-checkpoint /root/models/Qwen3-30B-A3B-Base \
  --save /root/models/Qwen3-30B-A3B-Base_torch_dist

Exact training profile

Parameter Value
Rollouts 100 (0..99)
Prompts per rollout 32
Responses per prompt 16
Trajectories per update 512
Global batch size 512
Train context / prompt / response caps 32,768 / 2,048 / 20,480
Packed tokens per GPU 65,536
Reward dapo, key score, correct +1, incorrect -1
Advantage estimator GRPO
Learning rate 1e-6, constant, 10 warmup iterations
PPO clipping low 0.20, high 0.28
Training parallelism TP4 / PP1 / CP1 / EP8 / ETP1
Rollout engine one 8-GPU vLLM engine
vLLM memory utilization 0.7
Evaluation AIME 2024 every 20 rollouts, 32 samples/problem, top-p 0.7
Checkpoints every 10 rollouts, retain interval 100
Actual launcher script

This is the task-owned launcher used for the Base-model run family. The completed run relied on the default full weight-sync mode; the preserved script now pins --update-weight-mode full explicitly so future defaults cannot change the reproduction. Override WORKTREE, data paths, checkpoint paths, W&B group, and Ray ports as needed.

#!/bin/bash
set -euo pipefail

pkill -9 -f "[v]llm serve|VLL[M]::" || true
pkill -9 -f "[t]rain.py|[t]rain_async.py|[M]egatronTrainRayActor|[R]olloutManager|multiprocessing.spawn" || true
sleep 3
ray stop --force || true
pkill -9 ray || true
pkill -9 vime || true
pkill -9 -f "[r]ay::" || true
sleep 3
pkill -9 ray || true
pkill -9 vime || true
pkill -9 redis || true

set -ex

export PYTHONUNBUFFERED=1
unset http_proxy https_proxy HTTP_PROXY HTTPS_PROXY

REPO_DIR="${WORKTREE:-/home/aoshen/vime/projects/vime-sync-slime/vime-review-worktree}"
cd "${REPO_DIR}"

source "${REPO_DIR}/scripts/models/qwen3-30B-A3B.sh"

RUN_STEPS="${RUN_STEPS:-100}"
TRAIN_DATA="${TRAIN_DATA:-/root/datasets/byted-dapo-official/dapo-math-17k-cleaned-vime.jsonl}"
EVAL_DATA="${EVAL_DATA:-/root/datasets/byted-dapo-official/aime-2024-cleaned-vime.jsonl}"
RM_TYPE="${RM_TYPE:-dapo}"
REWARD_KEY="${REWARD_KEY:-score}"
ROLLOUT_MAX_CONTEXT_LEN="${ROLLOUT_MAX_CONTEXT_LEN:-32768}"
ROLLOUT_MAX_PROMPT_LEN="${ROLLOUT_MAX_PROMPT_LEN:-2048}"
ROLLOUT_MAX_RESPONSE_LEN="${ROLLOUT_MAX_RESPONSE_LEN:-20480}"
ROLLOUT_BATCH_SIZE="${ROLLOUT_BATCH_SIZE:-32}"
N_SAMPLES_PER_PROMPT="${N_SAMPLES_PER_PROMPT:-16}"
GLOBAL_BATCH_SIZE="${GLOBAL_BATCH_SIZE:-512}"
LR_WARMUP_ITERS="${LR_WARMUP_ITERS:-10}"
EVAL_INTERVAL="${EVAL_INTERVAL:-20}"
EVAL_MAX_RESPONSE_LEN="${EVAL_MAX_RESPONSE_LEN:-20480}"
EVAL_SAMPLES_PER_PROMPT="${EVAL_SAMPLES_PER_PROMPT:-32}"
EVAL_TOP_P="${EVAL_TOP_P:-0.7}"
MAX_TOKENS_PER_GPU="${MAX_TOKENS_PER_GPU:-65536}"
VLLM_SERVER_CONCURRENCY="${VLLM_SERVER_CONCURRENCY:-32}"
VLLM_GPU_MEMORY_UTILIZATION="${VLLM_GPU_MEMORY_UTILIZATION:-0.7}"
RAY_PORT="${RAY_PORT:-26779}"
RAY_DASHBOARD_PORT="${RAY_DASHBOARD_PORT:-28276}"
RAY_DASHBOARD_AGENT_PORT="${RAY_DASHBOARD_AGENT_PORT:-52375}"
RAY_NODE_IP="${RAY_NODE_IP:-$(hostname -I | awk '{print $1}')}"
RAY_TEMP_DIR="${RAY_TEMP_DIR:-/tmp/ray-vime-qwen3-base-pilot}"
SAVE_DIR="${SAVE_DIR-/root/localckpt/qwen3-30b-a3b-base-pilot}"
BASE_LOAD_DIR="${BASE_LOAD_DIR:-/root/models/Qwen3-30B-A3B-Base_torch_dist}"
LOAD_DIR="${LOAD_DIR:-}"
WANDB_PROJECT="${WANDB_PROJECT:-vime-ab-30b-h200}"
WANDB_ENTITY="${WANDB_ENTITY:-aoshen524-purdue-university}"
WANDB_GROUP="${WANDB_GROUP:-qwen3-30b-a3b-base-dapo17k-canonical-pilot}"
MASTER_ADDR="${MASTER_ADDR:-127.0.0.1}"

if [[ -z "${LOAD_DIR}" && -n "${SAVE_DIR}" && -r "${SAVE_DIR}/latest_checkpointed_iteration.txt" ]]; then
   LOAD_DIR="${SAVE_DIR}"
fi

CKPT_ARGS=(
   --hf-checkpoint /root/models/Qwen3-30B-A3B-Base
   --ref-load "${BASE_LOAD_DIR}"
   --load "${LOAD_DIR:-${BASE_LOAD_DIR}}"
)
if [[ -z "${LOAD_DIR}" ]]; then
   CKPT_ARGS+=(--start-rollout-id 0)
else
   CKPT_ARGS+=(--override-opt-param-scheduler)
fi
if [[ -n "${SAVE_DIR}" ]]; then
   CKPT_ARGS+=(
      --save "${SAVE_DIR}"
      --save-interval 10
      --save-retain-interval 100
   )
fi

ROLLOUT_ARGS=(
   --prompt-data "${TRAIN_DATA}"
   --input-key prompt
   --label-key label
   --apply-chat-template
   --rollout-shuffle
   --rollout-seed 42
   --rm-type "${RM_TYPE}"
   --reward-key "${REWARD_KEY}"
   --log-passrate
   --num-rollout "${RUN_STEPS}"
   --rollout-batch-size "${ROLLOUT_BATCH_SIZE}"
   --n-samples-per-prompt "${N_SAMPLES_PER_PROMPT}"
   --rollout-max-context-len "${ROLLOUT_MAX_CONTEXT_LEN}"
   --rollout-max-prompt-len "${ROLLOUT_MAX_PROMPT_LEN}"
   --rollout-max-response-len "${ROLLOUT_MAX_RESPONSE_LEN}"
   --rollout-temperature 1
   --global-batch-size "${GLOBAL_BATCH_SIZE}"
   --balance-data
)

EVAL_ARGS=(
   --eval-interval "${EVAL_INTERVAL}"
   --eval-prompt-data aime "${EVAL_DATA}"
   --n-samples-per-eval-prompt "${EVAL_SAMPLES_PER_PROMPT}"
   --eval-max-context-len "${ROLLOUT_MAX_CONTEXT_LEN}"
   --eval-max-response-len "${EVAL_MAX_RESPONSE_LEN}"
   --eval-top-p "${EVAL_TOP_P}"
)

PERF_ARGS=(
   --tensor-model-parallel-size 4
   --sequence-parallel
   --pipeline-model-parallel-size 1
   --context-parallel-size 1
   --expert-model-parallel-size 8
   --expert-tensor-parallel-size 1
   --recompute-granularity full
   --recompute-method uniform
   --recompute-num-layers 1
   --calculate-per-token-loss
   --use-dynamic-batch-size
   --max-tokens-per-gpu "${MAX_TOKENS_PER_GPU}"
)

GRPO_ARGS=(
   --advantage-estimator grpo
   --kl-loss-coef 0.00
   --kl-loss-type low_var_kl
   --entropy-coef 0.00
   --eps-clip 0.2
   --eps-clip-high 0.28
)

OPTIMIZER_ARGS=(
   --optimizer adam
   --lr 1e-6
   --lr-warmup-iters "${LR_WARMUP_ITERS}"
   --lr-decay-style constant
   --weight-decay 0.1
   --adam-beta1 0.9
   --adam-beta2 0.98
   --optimizer-cpu-offload
   --overlap-cpu-optimizer-d2h-h2d
   --use-precision-aware-optimizer
)

WANDB_ARGS=(
   --use-wandb
   --wandb-project "${WANDB_PROJECT}"
   --wandb-team "${WANDB_ENTITY}"
   --wandb-group "${WANDB_GROUP}"
   --wandb-dir /root/runs/wandb
)

VLLM_ARGS=(
   --rollout-num-gpus 8
   --rollout-num-gpus-per-engine 8
   --vllm-gpu-memory-utilization "${VLLM_GPU_MEMORY_UTILIZATION}"
   --vllm-data-parallel-size 1
   --vllm-server-concurrency "${VLLM_SERVER_CONCURRENCY}"
   --vllm-all2all-backend deepep_high_throughput
   --vllm-moe-backend triton
)

MISC_ARGS=(
   --actor-num-nodes 1
   --actor-num-gpus-per-node 8
   --colocate
   --update-weight-mode full
   --bf16
   --attention-dropout 0.0
   --hidden-dropout 0.0
   --accumulate-allreduce-grads-in-fp32
   --attention-softmax-in-fp32
   --attention-backend flash
   --moe-token-dispatcher-type flex
   --moe-enable-deepep
   --train-memory-margin-bytes 2147483648
)

cleanup_ray() {
   pkill -TERM -f '^VLLM::' >/dev/null 2>&1 || true
   pkill -TERM -f '[M]egatronTrainRayActor|[R]olloutManager|[t]rain.py|[t]rain_async.py' >/dev/null 2>&1 || true
   sleep 2
   pkill -KILL -f '^VLLM::' >/dev/null 2>&1 || true
   ray stop --force >/dev/null 2>&1 || true
   pkill -KILL -f '[r]ay::|[M]egatronTrainRayActor|[R]olloutManager|[t]rain.py|[t]rain_async.py|multiprocessing.spawn' \
      >/dev/null 2>&1 || true
}
trap cleanup_ray EXIT

ray start \
   --head \
   --node-ip-address "${RAY_NODE_IP}" \
   --port="${RAY_PORT}" \
   --num-gpus 8 \
   --disable-usage-stats \
   --dashboard-host=0.0.0.0 \
   --dashboard-port="${RAY_DASHBOARD_PORT}" \
   --dashboard-agent-listen-port="${RAY_DASHBOARD_AGENT_PORT}" \
   --temp-dir="${RAY_TEMP_DIR}"

RUNTIME_ENV_JSON="{
  \"env_vars\": {
    \"PYTHONPATH\": \"${REPO_DIR}:/root/Megatron-LM/\",
    \"CUDA_DEVICE_MAX_CONNECTIONS\": \"1\",
    \"NCCL_NVLS_ENABLE\": \"0\",
    \"no_proxy\": \"127.0.0.1,${MASTER_ADDR}\",
    \"MASTER_ADDR\": \"${MASTER_ADDR}\"
  }
}"

ray job submit --address="http://127.0.0.1:${RAY_DASHBOARD_PORT}" \
   --runtime-env-json="${RUNTIME_ENV_JSON}" \
   -- python3 train.py \
   "${MODEL_ARGS[@]}" \
   "${CKPT_ARGS[@]}" \
   "${ROLLOUT_ARGS[@]}" \
   "${EVAL_ARGS[@]}" \
   "${OPTIMIZER_ARGS[@]}" \
   "${GRPO_ARGS[@]}" \
   "${WANDB_ARGS[@]}" \
   "${PERF_ARGS[@]}" \
   "${VLLM_ARGS[@]}" \
   "${MISC_ARGS[@]}"

Launch it with:

export WANDB_API_KEY='<from secret store>'
RUN_STEPS=100 \
SAVE_DIR=/root/localckpt/qwen3-30b-a3b-base-100step \
WANDB_GROUP=qwen3-30b-a3b-base-100step-repro \
bash run-qwen3-30B-A3B-base-pilot.sh

Notes

  • MAX_TOKENS_PER_GPU=65536 is the packed-token training budget, not a 64K per-sample response limit. Individual samples remain bounded by the 32K total context and 20K response cap.
  • The W&B API key is intentionally not included. Supply it through a secret environment variable or Buildkite secret store.
  • Do not merge until human review is complete. GPU validation is complete.

@read-the-docs-community

read-the-docs-community Bot commented Jul 12, 2026 •

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Gemma4 text models (dense and MoE), adds the SWE-bench Verified grading protocol, refactors agent harness execution to use a detached exec_and_wait mechanism, and implements a fused PPO log-probability and entropy calculation. The review feedback highlights several critical issues: a guaranteed crash in the attention forward pass due to an invalid enable_gqa argument in PyTorch's scaled_dot_product_attention; a potential NameError in the server's update handler under task cancellation; unsafe in-place tensor mutation in the PPO backward pass; a hardcoded 10-second sleep bottleneck in the generation loop; and potential cleanup failures in the Dockerfile and sandbox execution.

Comment on lines +667 to +668
else:
cu_seqlens_list = cu_seqlens.tolist()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

PyTorch's native torch.nn.functional.scaled_dot_product_attention does not accept an enable_gqa argument. Passing this argument will cause a guaranteed TypeError and crash the attention forward pass. PyTorch natively handles Grouped-Query Attention (GQA) broadcasting internally based on the head dimensions of the inputs.

                scale=scale,

Comment thread vime/agent/sandbox.py
env: dict[str, str] | None = None,
timeout: int = 120,
check: bool = False,
idempotent: bool = True,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-high high

The exec method (which runs arbitrary shell commands in the sandbox) defaults to idempotent=True. Arbitrary shell commands (such as process spawning or file appending) are generally not idempotent. Retrying them automatically on transient network errors can lead to dangerous double-execution side effects. It is safer to default idempotent to False and only explicitly set it to True for known read-only or safe commands.

Suggested change
idempotent: bool = True,
idempotent: bool = False,

Comment on lines 478 to +488
timeout_s = _get_update_timeout_s(payload, args)
update_state["in_progress"] = True
result = None
error = None
try:
before_loads = await _wait_until_idle(sample_manager, timeout_s)
update_result = await asyncio.to_thread(update_from_disk_fn, model_path)
after_loads = await _ray_get(sample_manager.get_loads.remote())
except TimeoutError as e:
return _json_error(str(e), 503)
error = {"ok": False, "status": 503, "error": str(e)}
except Exception as e:
return _json_error(f"update_from_disk failed: {e}", 500)
error = {"ok": False, "status": 500, "error": f"update_from_disk failed: {e}"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If the main update task is cancelled (e.g., due to a client disconnecting), an asyncio.CancelledError is raised. Since CancelledError inherits from BaseException (not Exception), it bypasses the except Exception blocks, leaving update_result unassigned. When the finally block is entered, constructing result with update_result will raise a NameError, masking the cancellation and potentially leaving coalesced tasks hung. Initializing the variables and catching BaseException to cleanly handle cancellation prevents this.

        timeout_s = _get_update_timeout_s(payload, args)
        result = None
        error = None
        before_loads = None
        after_loads = None
        update_result = None
        try:
            before_loads = await _wait_until_idle(sample_manager, timeout_s)
            update_result = await asyncio.to_thread(update_from_disk_fn, model_path)
            after_loads = await _ray_get(sample_manager.get_loads.remote())
        except TimeoutError as e:
            error = {"ok": False, "status": 503, "error": str(e)}
        except Exception as e:
            error = {"ok": False, "status": 500, "error": f"update_from_disk failed: {e}"}
        except BaseException as e:
            error = {"ok": False, "status": 500, "error": f"update_from_disk interrupted: {type(e).__name__}"}
            raise

Comment thread vime/utils/ppo_utils.py
grad_entropy_input.mul_(grad_entropy.reshape(-1, 1))

vocab_parallel_size = log_prob_softmax.size(-1)
grad_input = log_prob_softmax.neg_()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using log_prob_softmax.neg_() mutates a saved tensor in-place during the backward pass. In PyTorch, mutating saved tensors in-place is highly discouraged as it can corrupt the autograd history, break double-backward/gradcheck, or raise runtime errors. Using the out-of-place .neg() is safer and avoids side effects.

Suggested change
grad_input = log_prob_softmax.neg_()
grad_input = log_prob_softmax.neg()

Comment thread docker/Dockerfile
Comment on lines 45 to +49
RUN git clone https://github.com/Dao-AILab/flash-attention.git && \
cd flash-attention/ && git checkout fbf24f67cf7f6442c5cfb2c1057f4bfc57e72d89 && git submodule update --init && cd hopper/ && \
MAX_JOBS=96 python setup.py install && \
export python_path=`python -c "import site; print(site.getsitepackages()[0])"` && \
mkdir -p $python_path/flash_attn_3 && \
cp flash_attn_interface.py $python_path/flash_attn_3/flash_attn_interface.py && \
rm -rf flash-attention/
cd flash-attention/ && git checkout 002cce0a1068f8c07dfccb5a1d232b9a3276947c && git submodule update --init && \
cd hopper/ && \
FLASH_ATTENTION_FORCE_BUILD=TRUE MAX_JOBS=96 pip -v install . --no-build-isolation && \
cd /root/ && rm -rf flash-attention/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The flash-attention repository is cloned in the current working directory, but the cleanup command cd /root/ && rm -rf flash-attention/ attempts to delete it relative to /root/. If the working directory is not /root/, this leaves the cloned repository intact, bloating the Docker image. Deleting it relative to its parent directory avoids this issue.

RUN git clone https://github.com/Dao-AILab/flash-attention.git && \
    cd flash-attention/ && git checkout 002cce0a1068f8c07dfccb5a1d232b9a3276947c && git submodule update --init && \
    cd hopper/ && \
    FLASH_ATTENTION_FORCE_BUILD=TRUE MAX_JOBS=96 pip -v install . --no-build-isolation && \
    cd ../.. && rm -rf flash-attention/

Comment on lines +282 to +283
await state.adapter.drop_session(session_id, wait_timeout=30) # cleanup only, idempotent
await asyncio.sleep(10)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

A hardcoded 10-second sleep in the finally block of generate() will block the completion of every single generation/rollout task. In reinforcement learning training loops with thousands of rollouts, this introduces a massive throughput bottleneck. Consider removing this sleep or making it conditional/configurable.

Suggested change
await state.adapter.drop_session(session_id, wait_timeout=30) # cleanup only, idempotent
await asyncio.sleep(10)
await state.adapter.drop_session(session_id, wait_timeout=30) # cleanup only, idempotent

@aoshen02
aoshen02 marked this pull request as ready for review July 12, 2026 06:57
aoshen02 added 9 commits July 12, 2026 14:13
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
@aoshen02
aoshen02 force-pushed the codex/slime-2185-vime-review branch from 7f23d79 to 02df2fe Compare July 12, 2026 14:13
@aoshen02

Copy link
Copy Markdown
Collaborator Author
image The ci is actually work.

@aoshen02 aoshen02 changed the title sync: update vime through slime #2185 (review) weekly sync: update vime through slime #2185 (review) Jul 14, 2026
@aoshen02 aoshen02 changed the title weekly sync: update vime through slime #2185 (review) weekly sync: update vime through slime #2185 Jul 14, 2026
@CalvinXKY

Copy link
Copy Markdown
Collaborator

Overall: LGTM as the #338 redo with stronger validation.
Please re-trigger CI on tip, HEAD currently shows buildkite canceled, failure status.

@aoshen02

Copy link
Copy Markdown
Collaborator Author

Overall: LGTM as the #338 redo with stronger validation. Please re-trigger CI on tip, HEAD currently shows buildkite canceled, failure status.

The ci 610 failed for I don't know what reason, but ci 609 does work, I post the pictures there:
image

@CalvinXKY
CalvinXKY merged commit 8d8f255 into vllm-project:main Jul 14, 2026
3 checks passed
aoshen02 added a commit to aoshen02/vime that referenced this pull request Jul 15, 2026
* sync: mechanically merge slime through 680824dd

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* sync: adapt slime changes for vLLM

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* ci: register synced CPU and GPU tests

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* update

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* update

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* update

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* sync: tighten vLLM adaptations after review

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* fix: preserve vLLM top-p fallback

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* sync: complete slime adaptation audit

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* fix(ci): update external PD test for native vLLM

* fix(ci): use NIXL default UCX device selection

* fix(ci): configure Qwen3.6 Mamba state layout

Signed-off-by: aoshen02 <aoshen@inferact.ai>

* fix(ci): honor explicit NCCL NVLS override

* fix(ci): preserve explicit UCX device override

---------

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants