fix(vllm): don't stream raw tool-call markup as content when a tool parser is active

[pos-ei-don]

Description

When the vLLM backend has a tool_parser configured (options: [tool_parser:...]) and a request includes tools, streaming responses leak the model's raw tool-call markup as assistant content.

The streaming loop yields every text delta as delta.content:

if streaming:
    delta_iteration_text = iteration_text.removeprefix(generated_text)
    yield backend_pb2.Reply(
        message=bytes(delta_iteration_text, encoding='utf-8'),
        chat_deltas=[backend_pb2.ChatDelta(content=delta_iteration_text)],
    )

But the tool parser (extract_tool_calls) only runs on the full text after the stream completes. So for a model that emits e.g. <tool_call>{"name": ...}</tool_call>, the client receives that raw markup as delta.content chunks during the stream, and the structured tool_calls only arrive in the final chunk.

Reproduce (a tool parser such as qwen3_coder, streaming request with tools):

• Before: delta.content chunks contain <tool_call>..., finish_reason: stop.
• After: delta.tool_calls chunks with name/arguments, finish_reason: tool_calls.

Verified on NVIDIA GB10 / arm64 / CUDA 13, vLLM 0.23.0, with the qwen3_coder tool parser driving a coding agent over the streaming OpenAI API.

Fix

Buffer the text while a tool parser is active for the request, and let the existing end-of-stream ChatDelta carry the parsed tool_calls (which the Go side already converts to SSE delta.tool_calls via ToolCallsFromChatDeltas / buildDeferredToolCallChunks). When the parser finds no tool call, flush the buffered content as a single content delta before the final chunk. Non-tool-parser streaming is unchanged.

[pos-ei-don]

Updated after end-to-end verification on the live backend (NVIDIA GB10 / arm64 / CUDA 13, vLLM 0.23.0, qwen3_coder tool parser, streaming over the OpenAI API):

• Tool-call case ✅ — delta.tool_calls with name/arguments, finish_reason: tool_calls, no raw markup in content.
• Plain-text-with-tools case — testing surfaced a second facet of the same bug: when a tool is offered but the model answers in plain text, the buffered content was being emitted twice (once via an extra flush, once via the final chat_delta which already carries it). Fixed by dropping the redundant flush — the existing end-of-stream chat_delta.content is the single source. Re-verified: content now arrives exactly once.

Also added a server-less regression test (test_streaming_tool_parser_buffering) that mocks the engine + tool parser and asserts both invariants: no raw tool-call markup leaks as streamed content, and buffered content is delivered exactly once. It fails on the pre-fix code and passes on this change.

[pos-ei-don]

Superseded by #10351, which goes one step further: instead of buffering the entire generation, it delegates to parser.extract_tool_calls_streaming (implemented by every vLLM 0.23+ tool parser) and streams progressively. The buffer-all path becomes the fallback for parsers that lack the streaming method (none, today, but defensive).

Same correctness as this PR (no raw tool-call markup in delta.content), plus restored streaming for plain-text-with-tools responses. Verified end-to-end against qwen3_coder — plain-text TTFT drops from ~18s (long output) to ~0.22s.

Closing in favour of #10351.

[pos-ei-don]

Superseded by #10351, which takes a better approach: progressive streaming via the parser's extract_tool_calls_streaming (token-by-token) instead of buffering the whole output, and it also covers the plain-text-with-tools content-duplication edge case. Please disregard the update above — continuing in #10351.