[Bug]: OpenAI Responses API - Tool Call After Thinking Fails with 400 Error

[RiccardoRubini]

Bug Description

Hello,
I'm having issues with the OpenAI Responses API when combining thinking with tool calls.

Setup:

from llama_index.llms.openai.responses import OpenAIResponses 

llm = OpenAIResponses(
    model='gpt-5',
    api_base=AZURE_AI_FOUNDRY_ENDPOINT_OPENAILIKE,
    api_key=AZURE_AI_FOUNDRY_KEY,
    max_tokens=max_tokens,
    timeout=timeout,
    reasoning_options={"effort": "high", "summary": "auto"}
)
Settings.llm = llm

Problem:
The model works fine when it only thinks OR only calls a tool. But when a tool call follows thinking, I get this error:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Item with id 'rs_03d84ac1087bdc8c01693036340724819783adfd5c3224f826' not found. Items are not persisted when `store` is set to false. Try again with `store` set to true, or remove this item from your input.", 'type': 'invalid_request_error', 'param': 'input', 'code': None}}

Attempted fix:
Setting store=True (even if I do not want to store any messages) gives a different error:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Item 'rs_0cf94f2fa7f56d54006930382e84b081909b6464fc7cddd4b7' of type 'reasoning' was provided without its required following item.", 'type': 'invalid_request_error', 'param': 'input', 'code': None}}

I've found some related issues in other projects: this and this

Regards,
RR

Version

core: 0.14.8 - llm-openai: 0.6.9

Steps to Reproduce

See the issue

Relevant Logs/Tracbacks

[dosubot]

Hi @RiccardoRubini! I'm Dosu and I’m helping the llama_index team.

This is a limitation of how OpenAIResponses in LlamaIndex interacts with the OpenAI Responses API. When store=False (the default), reasoning steps (like "thinking") are not persisted server-side, so if a subsequent tool call references that reasoning step by ID, you get a 400 error about the item not being found. When you set store=True, the API expects the reasoning item to have a required follow-up (like encrypted content or a summary); if that's missing, you get a different 400 error about missing follow-up items. The LlamaIndex wrapper just forwards the store flag and doesn't handle these edge cases or rehydrate missing items, so chaining reasoning and tool calls without storing everything isn't currently supported in this integration. There are no built-in workarounds or recent fixes for this in LlamaIndex as of now. For more details, see the OpenAIResponses implementation.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Prozac614]

Hi @RiccardoRubini , thanks for reporting this issue with detailed reproduction steps!

I'd like to work on this.

After analyzing the codebase, I found the root cause is in to_openai_responses_message_dict() function in utils.py. When reasoning items are serialized for the API input, they're sent as ID references only, which causes issues because:

1. With store=False: OpenAI doesn't persist reasoning items, so ID references fail
2. With store=True: Reasoning items need to be properly paired with their following items

Proposed fix: Instead of sending reasoning items as ID references, we should either:

• Omit reasoning items from the conversation history input entirely (since they represent internal model thinking)
• Or properly structure the reasoning items with their required following items according to OpenAI's API spec

@logan-markewich What's your preferred approach here?