[BUG] Stop does not work on OpenAI Compatible API Provider

[BluePeer]

Problem (one or two sentences)

I use a llama.cpp router configured setup.
config in roo code use it as OpenAI Compatible Api Provider
If you send a Promt you are currently not able to Stop the Inference on server if you Press the UI Stop Button.

It looks like a U / Roo Code Internal issue, not sending the request to the llama.cpp server.
the General OpenAI Api Timeout (On long Promt Processings for example) works, the Roo Code emit a Stop, and the Server Stop the Processing and Restart with the fresh Inited new Processing.
But on Press the Stop button, Roo Code does not Emit the stop to the API and leave the llama.cpp server with a Useless running Inference. tested with different backends like LM Studio and llama.cpp, shoes alltime the same case timeout related stop requests from Roo Code Internals, stop the Inference Processing, but Manual Stop Button Press Not.

Context (who is affected and when)

• OpenAI Compatible API Provider (Custom Base URL)
• Multiple Backends (Only tested with llama.cpp and LM Studio)

Reproduction steps

Send a Promt to a OpenAI Compatible custom Base URL and press the Stop Button on Chat Input.
The UI Shows stop, the Inference still run (Not get a Stop request)

Expected result

emit the Stop request to the API

Actual result

inference still run, the API get no stop request.

Variations tried (optional)

No response

App Version

3.41.1

API Provider (optional)

OpenAI Compatible

Model Used (optional)

any models

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

[roomote]

I've analyzed this issue and here's my proposed implementation plan:

Root Cause

The OpenAiHandler class (used for OpenAI-compatible providers like llama.cpp, LM Studio, etc.) does not implement AbortController support for request cancellation. When the user presses the Stop button:

1. The Task class sets this.abort = true and calls cancelCurrentRequest()
2. However, the OpenAiHandler does NOT pass an abort signal to the underlying OpenAI SDK calls
3. The server continues generating tokens because it never receives a cancellation signal

In contrast, the OpenAiNativeHandler class properly implements abort support by:

• Creating an AbortController at the start of createMessage()
• Passing signal: this.abortController.signal to API requests
• Checking this.abortController.signal.aborted in the stream loop

Proposed Fix

Add AbortController support to src/api/providers/openai.ts (OpenAiHandler class):

1. Add abort controller property:

   private abortController?: AbortController

2. In createMessage() method, create the controller before making requests:

   this.abortController = new AbortController()

3. Pass the signal to SDK calls:

   stream = await this.client.chat.completions.create(
     requestOptions,
     { signal: this.abortController.signal, ...otherOptions }
   )

4. Check for abort in the stream loop:

   for await (const chunk of stream) {
     if (this.abortController?.signal.aborted) {
       break
     }
     // ... process chunk
   }

5. Clean up the controller:

   finally {
     this.abortController = undefined
   }

6. Update completePrompt() method with the same abort signal pattern.

7. Also apply the same fix to BaseOpenAiCompatibleProvider in src/api/providers/base-openai-compatible-provider.ts since it has the same issue and is used by many other providers (Groq, SambaNova, Fireworks, etc.).

This approach mirrors the implementation in OpenAiNativeHandler and will allow the HTTP connection to be properly terminated when the user presses Stop, signaling the server to abort inference.

Please mention @roomote in a comment saying that you approve this plan, and I'll proceed with the implementation.

[BluePeer]

@roomote that looks good, attempt fix

[roomote]

Thanks for the approval! I will now implement the fix by adding AbortController support to the OpenAiHandler and BaseOpenAiCompatibleProvider classes. I'll open a PR shortly to share progress.

[roomote]

Opened PR #10784 with my attempt at addressing this. The fix adds AbortController support to the OpenAiHandler and BaseOpenAiCompatibleProvider classes so that the Stop button properly cancels requests for OpenAI-compatible providers. Feedback welcome!