fix!: Enhance response API support to not fail with tool calling

[luis5tb]

What does this PR do?

Introduces two main fixes to enhance the stability of Responses API when dealing with tool calling responses and structured outputs.

Changes Made

1. It added OpenAIResponseOutputMessageMCPCall and ListTools to OpenAIResponseInput but feat(responses)!: improve responses + conversations implementations #3810 got merge that did the same in a different way. Still this PR does it in a way that keep the sync between OpenAIResponsesOutput and the allowed objects in OpenAIResponseInput.

2. Add protection in case self.ctx.response_format does not have type attribute

BREAKING CHANGE: OpenAIResponseInput now uses OpenAIResponseOutput union type.
This is semantically equivalent - all previously accepted types are still supported
via the OpenAIResponseOutput union. This improves type consistency and maintainability.

[jaideepr97]

hi @luis5tb would you mind elaborating a little on what the exact issue you observed is and how your PR addresses it?

[ashwinb]

as @jaideepr97 noted, the PR summary is not quite sufficient to justify the substantial changes particularly to the API shape also.

[luis5tb]

Thanks! Yes, I'll update the description with more information. The TLDR is that when using GPT-OSS model (with remote-vllm) I'm getting a lot of issues related to the conversion/generation of Messages between VLLM, OpenAI and LlamaStack formats.

Regarding the API changes:

• I found that after calling an MCP tool and calling again the vllm inference endpoint with the previous responses ID, there was a need to include the OpenAIResponseOutputMessageMCPCall/ListTools as a possible OpenAIResponseInput for that request, otherwise there is an error
• The modification on OpenAIToolMessageParam was because at the responses/tool_executor, when building the input_message we are adding OpenAIChatCompletionContentPartImageParam, so it is not just text. This actually still have problems with the nested objects so I make it str too (at least for now)

The other changes are basically to handle cases that probably were not considered and may be due to final implementation on VLLM and the objects it returns. All the changes are related to ensuring the format of the messages can be parsed by the different components (vllm and llamastack). Perhaps the base problem comes from LlamaStack translating the responses API calls to VLLM into chat_completion API calls, and that makes the message parsing a bit difficult

[jwm4]

@luis5tb Are you still planning to update the PR description?

[luis5tb]

@luis5tb Are you still planning to update the PR description?

Yes! I was testing the pieces that are not needed anymore due to changes in vLLM and what else what needed due to new changes in too. I'll update later today or first thing tomorrow morning

[luis5tb]

@jwm4 @ashwinb @jaideepr97 Let me know if the current description is enough or more/different details are needed

[jaideepr97]

still seems to be failing integration tests though
@luis5tb since you're changing some datatypes you will need to update the api conformance test as well

[luis5tb]

still seems to be failing integration tests though @luis5tb since you're changing some datatypes you will need to update the api conformance test as well

Integration test fixed.

Regarding the api conformance tests, I'm not entirely sure how to update/fix those. Can you give me some hints? I thought it was about executing the script to update the openapi spec (run_openapi_generator.sh). Note this change is only adding new types supported as input, not removing the existing ones, so there is no breaking on the API, was what accepted is still accepted. Only difference is that a couple of other types are accepted too

[jaideepr97]

since you're changing some datatypes you will need to update the api conformance test as well

Regarding the api conformance tests, I'm not entirely sure how to update/fix those. Can you give me some hints? I thought it was about executing the script to update the openapi spec (run_openapi_generator.sh). Note this change is only adding new types supported as input, not removing the existing ones, so there is no breaking on the API, was what accepted is still accepted. Only difference is that a couple of other types are accepted too

I guess it is technically a breaking change in the spec since you're removing fields, even though you are replacing them with the parent object. The test seems to be directly comparing removals and additions

error	[request-property-one-of-removed] at docs/static/llama-stack-spec.yaml	
	in API POST /v1/responses
		removed '#/components/schemas/OpenAIResponseOutputMessageWebSearchToolCall, #/components/schemas/OpenAIResponseOutputMessageFileSearchToolCall, #/components/schemas/OpenAIResponseOutputMessageFunctionToolCall, #/components/schemas/OpenAIResponseMCPApprovalRequest' from the 'input/oneOf[subschema #2]/items/' request property 'oneOf' list

though I'm not sure why it also errors out on new additions
@cdoern maybe able to advise here

[cdoern]

@jaideepr97 these types of changes are ok, just prefix the title of the PR with fix!: or put BREAKING CHANGE: ... with a description in the commit and the test will pass.

[luis5tb]

@jaideepr97 these types of changes are ok, just prefix the title of the PR with fix!: or put BREAKING CHANGE: ... with a description in the commit and the test will pass.

Thanks! I updated the PR with that prefix + breaking change information

[luis5tb]

as @jaideepr97 noted, the PR summary is not quite sufficient to justify the substantial changes particularly to the API shape also.

done

[ashwinb]

lgtm