fix: add mergeToolResultText for OpenAI provider to fix Mistral tool message ordering #10695
+11
−4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #10684
Related: #10618, #10540 (same root cause, different providers)
Description
This PR attempts to address Issue #10684. Feedback and guidance are welcome.
Root Cause:
The error "Unexpected role 'user' after role 'tool'" occurs because Mistral/Devstral models (and other strict OpenAI-compatible servers like vLLM, llama.cpp, Ollama) enforce strict message ordering. After a
toolrole message, the next message cannot be auserrole message.In the current code, when a user message contains both tool results and text content (like environment_details), the conversion logic:
toolrole messages for each tool resultuserrole message for any text contentThis creates the invalid sequence:
tool->user, which strict servers reject.Solution:
Added
{ mergeToolResultText: true }to allconvertToOpenAiMessages()calls in the OpenAI provider. This option merges text content into the last tool message instead of creating a separate user message, avoiding the invalid message sequence.This uses the same
mergeToolResultTextmechanism that:Test Procedure
mergeToolResultTexttests inopenai-format.spec.tscover the underlying functionalityPre-Submission Checklist
mergeToolResultTextfunctionality is already tested inopenai-format.spec.ts.Documentation Updates
Additional Notes
This is a minimal fix that leverages existing infrastructure. The
mergeToolResultTextoption is already well-tested and used by other providers (DeepSeek, Z.ai, MiniMax) for similar message ordering requirements.Important
Fixes message ordering in OpenAiHandler by merging tool result text to prevent invalid sequences in strict servers.
{ mergeToolResultText: true }toconvertToOpenAiMessages()calls inOpenAiHandlerto fix message ordering issues.tool->usermessage sequences in strict OpenAI-compatible servers (e.g., vLLM, llama.cpp, Ollama).openai.tsto applymergeToolResultTextin both streaming and non-streaming modes.createMessage()andhandleO3FamilyMessage()functions.openai-format.spec.tsformergeToolResultTextfunctionality.This description was created by
for 6cab7c1. You can customize this summary. It will automatically update as commits are pushed.