Skip to content

feat(openai): option to send the Responses message phase when the item starts - #8429

Draft
cchen-dialpad wants to merge 1 commit into
livekit:mainfrom
cchen-dialpad:feat/responses-phase-on-item-added
Draft

cchen-dialpad wants to merge 1 commit into
livekit:mainfrom
cchen-dialpad:feat/responses-phase-on-item-added

Conversation

@cchen-dialpad

Copy link
Copy Markdown

Summary

The OpenAI Responses API sets an output message's phase (commentary or final_answer) on response.output_item.added. ResponsesLLMStream ignores that event and sends the phase only from response.output_item.done, after the message's text. An agent that speaks only the final_answer therefore has to buffer each message until it ends.

This PR adds an opt-in setting, openai.responses.LLM(emit_phase_on_item_start=True):

  • The phase chunk (delta.extra == {"openai": {"phase": ...}}) is sent when the message starts, before its text, and isn't repeated when it ends.
  • If the start event has no phase, the phase is still sent when the message ends.
  • With the default (False), behaviour is unchanged: output_item.added is still ignored on both WebSocket and HTTP.

ResponsesLLMStream takes a matching optional argument with the same default, so inference.responses.LLM and the Azure, xAI and Perplexity subclasses are unaffected.

Usage

llm = openai.responses.LLM(emit_phase_on_item_start=True)

class MyAgent(Agent):
    async def llm_node(self, chat_ctx, tools, model_settings):
        phase = None
        async for chunk in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
            if isinstance(chunk, llm.ChatChunk) and chunk.delta:
                phase = (chunk.delta.extra or {}).get("openai", {}).get("phase") or phase
                if chunk.delta.content and phase == "commentary":
                    continue
            yield chunk

Tests

New hermetic tests in tests/test_plugin_openai_responses.py, run over both WebSocket and HTTP:

  • With the setting unset or False, the phase arrives after the text.
  • With it True, each phase arrives once, before its text.
  • With it True but no phase on the start event, the phase arrives after the text.

make check and pytest --plugin openai tests/test_plugin_openai_responses.py pass.

🤖 Generated with Claude Code

…m starts

The Responses API sets an output message's phase (commentary or
final_answer) on response.output_item.added, but the engine ignored that
event and sent the phase only from response.output_item.done, after the
message's text. A caller that speaks only the final_answer had to buffer
each message until it ended.

openai.responses.LLM(emit_phase_on_item_start=True) sends the phase chunk
when the message starts, before its text, and does not repeat it when the
message ends. If the start event carries no phase, it is still sent at the
end. The default (False) keeps the current behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@longcw

longcw commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

related to #8427

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants