🤖 fix: detect and abort repetitive Gemini output to prevent token exhaustion #828
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
src/node/services/streamManager.ts
Outdated
| // Abort via the controller - the stream loop will detect this and exit | ||
| streamInfo.abortController.abort(); | ||
| break; |
There was a problem hiding this comment.
When the repetition detector fires, it simply aborts the controller and breaks out of the text-delta handler, but processStreamWithCleanup only emits stream-end when the abort signal is false and there is no stream-abort emission for this path. That means a Gemini loop termination produces no completion event at all, so AIService never commits the partial and front-end listeners wait indefinitely for a terminal update. Consider emitting stream-abort (or an error) when the detector cancels the stream.
Useful? React with 👍 / 👎.
…austion This addresses the known Gemini token exhaustion bug where the model gets stuck in a loop emitting variations like 'I am done. I am done. I am done...' until it exhausts all output tokens. The fix: - Adds RepetitionDetector class that monitors streaming text for repetitive patterns using a sliding window approach - Integrates detector into StreamManager for Gemini models only - Automatically aborts the stream when 10+ repetitions of the same phrase (8-50 chars) are detected within a 2000 char window This is the same approach used by gemini-cli for loop detection. See: google-gemini/gemini-cli#13322 _Generated with `mux`_
ammar-agent
force-pushed
the
fix-gemini-token-exhaustion-bug
branch
from
0a03884 to
c8366c6
Compare
|
Closed for gross |
2 participants
Summary
This addresses the known Gemini token exhaustion bug where the model gets stuck in a loop emitting variations like
I am done. I am done. I am done...until it exhausts all output tokens.The Bug
This is a well-documented issue with Gemini models:
The gemini-cli implements loop detection to catch this, and now we do too.
The Fix
RepetitionDetectorclass that monitors streaming text for repetitive patterns using a sliding window approachTesting
RepetitionDetectorclass covering:I am done. I am done.)addTextcallsGenerated with
mux