Skip to content

fix(template): align thinking-history and enable_thinking defaults with official templates - #10589

Open
hjh0119 wants to merge 10 commits into
modelscope:mainfrom
hjh0119:fix-nemotron-template
Open

hjh0119 wants to merge 10 commits into
modelscope:mainfrom
hjh0119:fix-nemotron-template

Conversation

@hjh0119

@hjh0119 hjh0119 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes the thinking-history handling and default thinking mode so that rendered prompts match the
official chat templates.

  • A bare tool message no longer ends the current round: only user, or a tool turn carrying a
    user follow-up, starts a new round, so the assistant that issued a tool call keeps its reasoning.
  • DeepSeek V3.1 keeps tool-call turns verbatim while stripping plain history turns; V4/V4.1 keep
    the same rule and </think> trailing whitespace is now controlled by strip_history_thinking.
  • Kimi K2.5 treats the last non-tool-call assistant as history, so its reasoning is truncated like
    the official jinja.
  • Hy3 keeps history reasoning when tools are provided and strips it otherwise, matching the
    official template.
  • Empty think blocks are placed where the official templates put them: qwen3 / MiniCPM-o / MiniCPM-V
    only on the final response, DeepSeek V3.1 / Gemma4 / diffusion_gemma only right after a user turn.
  • enable_thinking defaults now follow the official templates: 21 thinking templates default to
    thinking, and the MiniCPM5 family becomes auto (no prefix, no switch, model decides).
  • Replaces the non_thinking_prefix_only_after_user class attribute with TemplateMeta policies
  • Adds policy unit tests plus a default-value parity check against local official templates.

Breaking changes

  • Multi-turn and tool-call inference prompts change: reasoning is kept across tool calls.
  • Default inference mode flips to thinking for 21 templates (Hunyuan's default tier and
    DeepSeek-V4's default reasoning_effort change with it).
  • Training samples change when loss_scale=last_round or preserve_thinking=False: history
    trimming, empty-think insertion, and the Kimi / Hy3 branches differ.

@hjh0119 hjh0119 changed the title fix(template): fix preserve_thinking fix(template): align thinking-history and enable_thinking defaults with official templates Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants