Skip to content

Add configurable reasoning preservation for openai completions #268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zikajk wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Add configurable reasoning preservation for openai completions #268

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
20579ec
add configurable reasoning preservation for openai completions
zikajk Jan 15, 2026
17623a4
fix tests and improve docs
zikajk Jan 17, 2026
c2d8af8
Replace keepHistoryReasoning boolean with reasoningHistory
zikajk Jan 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## Unreleased

- (OpenAI Chat) - Configurable reasoning history via `reasoningHistory` (model-level, default: all)

## 0.94.1

- Fix tools prompt override not working via config.
Expand Down
3 changes: 2 additions & 1 deletion docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -693,7 +693,8 @@ To configure, add your OTLP collector config via `:otlp` map following [otlp aut
thinkTagEnd?: string;
models: {[key: string]: {
modelName?: string;
extraPayload?: {[key: string]: any}
extraPayload?: {[key: string]: any};
reasoningHistory?: "all" | "turn" | "off";
}};
}};
defaultModel?: string;
Expand Down
42 changes: 28 additions & 14 deletions docs/models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,19 +61,20 @@ You just need to add your provider to `providers` and make sure add the required

Schema:

| Option | Type | Description | Required |
|-------------------------------|--------|--------------------------------------------------------------------------------------------------------------|----------|
| `api` | string | The API schema to use (`"openai-responses"`, `"openai-chat"`, or `"anthropic"`) | Yes |
| `url` | string | API URL (with support for env like `${env:MY_URL}`) | No* |
| `key` | string | API key (with support for `${env:MY_KEY}` or `{netrc:api.my-provider.com}` | No* |
| `completionUrlRelativePath` | string | Optional override for the completion endpoint path (see defaults below and examples like Azure) | No |
| `thinkTagStart` | string | Optional override the think start tag tag for openai-chat (Default: "<think>") api | No |
| `thinkTagEnd` | string | Optional override the think end tag for openai-chat (Default: "</think>") api | No |
| `httpClient` | map | Allow customize the http-client for this provider requests, like changing http version | No |
| `models` | map | Key: model name, value: its config | Yes |
| `models <model> extraPayload` | map | Extra payload sent in body to LLM | No |
| `models <model> modelName` | string | Override model name, useful to have multiple models with different configs and names that use same LLM model | No |
| `fetchModels` | boolean | Enable automatic model discovery from `/models` endpoint (OpenAI-compatible providers) | No |
| Option | Type | Description | Required |
|---------------------------------------|---------|--------------------------------------------------------------------------------------------------------------|----------|
| `api` | string | The API schema to use (`"openai-responses"`, `"openai-chat"`, or `"anthropic"`) | Yes |
| `url` | string | API URL (with support for env like `${env:MY_URL}`) | No* |
| `key` | string | API key (with support for `${env:MY_KEY}` or `{netrc:api.my-provider.com}` | No* |
| `completionUrlRelativePath` | string | Optional override for the completion endpoint path (see defaults below and examples like Azure) | No |
| `thinkTagStart` | string | Optional override the think start tag tag for openai-chat (Default: "<think>") api | No |
| `thinkTagEnd` | string | Optional override the think end tag for openai-chat (Default: "</think>") api | No |
| `httpClient` | map | Allow customize the http-client for this provider requests, like changing http version | No |
| `models` | map | Key: model name, value: its config | Yes |
| `models <model> extraPayload` | map | Extra payload sent in body to LLM | No |
| `models <model> modelName` | string | Override model name, useful to have multiple models with different configs and names that use same LLM model | No |
| `models <model> reasoningHistory` | string | Controls reasoning in conversation history: `"all"` (default), `"turn"`, or `"off"` | No |
| `fetchModels` | boolean | Enable automatic model discovery from `/models` endpoint (OpenAI-compatible providers) | No |

_* url and key will be searched as envs `<provider>_API_URL` and `<provider>_API_KEY`, they require the env to be found or config to work._

Expand Down Expand Up @@ -120,6 +121,19 @@ Examples:

This way both will use gpt-5 model but one will override the reasoning to be high instead of the default.

=== "Reasoning in conversation history"
`reasoningHistory` - Controls whether and how the model's reasoning (thinking blocks, reasoning_content) is included in conversation history sent to the model.

**Available modes:**

- **`"all"`** (default, safe choice) - Send all reasoning blocks back to the model. The model can see its full chain of thought from previous turns. This is the safest option.
- **`"turn"`** - Send only reasoning from the current conversation turn (after the last user message). Previous reasoning is discarded before sending to the API.
- **`"off"`** - Never send reasoning blocks to the model. All reasoning is discarded before API calls.

**Note:** Reasoning is always shown to you in the UI and stored in chat history—this setting only controls what gets sent to the model in API requests.

Default: `"all"`.

=== "Dynamic model discovery"

For OpenAI-compatible providers, set `fetchModels: true` to automatically discover available models:
Expand Down Expand Up @@ -211,7 +225,7 @@ Notes:
3. Type the chosen method
4. Authenticate in your browser, copy the code.
5. Paste and send the code and done!

=== "Codex / Openai"

1. Login to Openai via the chat command `/login`.
Expand Down
3 changes: 2 additions & 1 deletion src/eca/config.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -336,7 +336,8 @@
{:kebab-case-key
[[:providers]]
:keywordize-val
[[:providers :ANY :httpClient]]
[[:providers :ANY :httpClient]
[:providers :ANY :models :ANY :reasoningHistory]]
:stringfy-key
[[:behavior]
[:providers]
Expand Down
5 changes: 5 additions & 0 deletions src/eca/llm_api.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@
provider-config (get-in config [:providers provider])
model-config (get-in provider-config [:models model])
extra-payload (:extraPayload model-config)
reasoning-history (or (:reasoningHistory model-config) :all)
[auth-type api-key] (llm-util/provider-api-key provider provider-auth config)
api-url (llm-util/provider-api-url provider config)
{:keys [handler]} (provider->api-handler provider config)
Expand All @@ -123,6 +124,7 @@
:web-search web-search
:extra-payload (merge {:parallel_tool_calls true}
extra-payload)
:reasoning-history reasoning-history
:api-url api-url
:api-key api-key
:auth-type auth-type}
Expand Down Expand Up @@ -157,6 +159,7 @@
:tools tools
:extra-payload (merge {:parallel_tool_calls true}
extra-payload)
:reasoning-history reasoning-history
:api-url api-url
:api-key api-key
:extra-headers {"openai-intent" "conversation-panel"
Expand All @@ -179,6 +182,7 @@
:tools tools
:think-tag-start "<thought>"
:think-tag-end "</thought>"
:reasoning-history reasoning-history
:extra-payload (merge {:parallel_tool_calls false}
(when reason?
{:extra_body {:google {:thinking_config {:include_thoughts true}}}})
Expand Down Expand Up @@ -221,6 +225,7 @@
:url-relative-path url-relative-path
:think-tag-start think-tag-start
:think-tag-end think-tag-end
:reasoning-history reasoning-history
:http-client http-client
:api-url api-url
:api-key api-key}
Expand Down
38 changes: 23 additions & 15 deletions src/eca/llm_providers/openai_chat.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,18 +384,26 @@
(reset! reasoning-state* {:id nil :type nil :content "" :buffer ""})))

(defn ^:private prune-history
"Ensure DeepSeek-style reasoning_content is discarded from history but kept for the active turn.
Only drops 'reason' messages WITH :delta-reasoning? before the last user message.
Think-tag based reasoning (without :delta-reasoning?) is preserved and transformed to assistant messages."
[messages]
(if-let [last-user-idx (llm-util/find-last-user-msg-idx messages)]
(->> messages
(keep-indexed (fn [i m]
(when-not (and (= "reason" (:role m))
(get-in m [:content :delta-reasoning?])
(< i last-user-idx))
m)))
vec)
"Discard reasoning messages from history based on reasoning-history mode.

Parameters:
- messages: the conversation history
- reasoning-history: controls reasoning retention
- :all - preserve all reasoning in history (safe default)
- :turn - preserve reasoning only in the current turn (after last user message)
- :off - discard all reasoning messages"
[messages reasoning-history]
(case reasoning-history
:all messages
:off (filterv #(not= "reason" (:role %)) messages)
:turn (if-let [last-user-idx (llm-util/find-last-user-msg-idx messages)]
(->> messages
(keep-indexed (fn [i m]
(when-not (and (= "reason" (:role m))
(< i last-user-idx))
m)))
vec)
messages)
messages))

(defn chat-completion!
Expand All @@ -406,14 +414,14 @@
Compatible with OpenRouter and other OpenAI-compatible providers."
[{:keys [model user-messages instructions temperature api-key api-url url-relative-path
past-messages tools extra-payload extra-headers supports-image?
think-tag-start think-tag-end http-client]}
think-tag-start think-tag-end reasoning-history http-client]}
{:keys [on-message-received on-error on-prepare-tool-call on-tools-called on-reason on-usage-updated] :as callbacks}]
(let [think-tag-start (or think-tag-start "<think>")
think-tag-end (or think-tag-end "</think>")
stream? (boolean callbacks)
system-messages (when instructions [{:role "system" :content instructions}])
;; Pipeline: prune history -> normalize -> merge adjacent assistants -> filter
all-messages (prune-history (vec (concat past-messages user-messages)))
all-messages (prune-history (vec (concat past-messages user-messages)) reasoning-history)
messages (vec (concat
system-messages
(normalize-messages all-messages supports-image? think-tag-start think-tag-end)))
Expand Down Expand Up @@ -473,7 +481,7 @@
tool-calls))
on-tools-called-wrapper (fn on-tools-called-wrapper [tools-to-call on-tools-called handle-response]
(when-let [{:keys [new-messages]} (on-tools-called tools-to-call)]
(let [pruned-messages (prune-history new-messages)
(let [pruned-messages (prune-history new-messages reasoning-history)
new-messages-list (vec (concat
system-messages
(normalize-messages pruned-messages supports-image? think-tag-start think-tag-end)))
Expand Down
55 changes: 48 additions & 7 deletions test/eca/llm_providers/openai_chat_test.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@
{:role "assistant" :reasoning_content "Thinking..."}])))))

(deftest prune-history-test
(testing "Drops reason messages WITH :delta-reasoning? before the last user message (DeepSeek)"
(testing "reasoningHistory \"turn\" drops all reason messages before the last user message"
(is (match?
[{:role "user" :content "Q1"}
{:role "assistant" :content "A1"}
Expand All @@ -272,12 +272,12 @@
{:role "assistant" :content "A1"}
{:role "user" :content "Q2"}
{:role "reason" :content {:text "r2" :delta-reasoning? true}}
{:role "assistant" :content "A2"}]))))
{:role "assistant" :content "A2"}]
:turn))))

(testing "Preserves reason messages WITHOUT :delta-reasoning? (think-tag based)"
(testing "reasoningHistory \"turn\" also drops think-tag reasoning before last user message"
(is (match?
[{:role "user" :content "Q1"}
{:role "reason" :content {:text "thinking..."}}
{:role "assistant" :content "A1"}
{:role "user" :content "Q2"}
{:role "reason" :content {:text "more thinking..."}}
Expand All @@ -288,12 +288,53 @@
{:role "assistant" :content "A1"}
{:role "user" :content "Q2"}
{:role "reason" :content {:text "more thinking..."}}
{:role "assistant" :content "A2"}]))))
{:role "assistant" :content "A2"}]
:turn))))

(testing "No user message leaves list unchanged"
(testing "reasoningHistory \"all\" preserves all reasoning"
(is (match?
[{:role "user" :content "Q1"}
{:role "reason" :content {:text "r1"}}
{:role "assistant" :content "A1"}
{:role "user" :content "Q2"}
{:role "reason" :content {:text "r2"}}
{:role "assistant" :content "A2"}]
(#'llm-providers.openai-chat/prune-history
[{:role "user" :content "Q1"}
{:role "reason" :content {:text "r1"}}
{:role "assistant" :content "A1"}
{:role "user" :content "Q2"}
{:role "reason" :content {:text "r2"}}
{:role "assistant" :content "A2"}]
:all))))

(testing "reasoningHistory \"off\" removes all reasoning messages"
(is (match?
[{:role "user" :content "Q1"}
{:role "assistant" :content "A1"}
{:role "user" :content "Q2"}
{:role "assistant" :content "A2"}]
(#'llm-providers.openai-chat/prune-history
[{:role "user" :content "Q1"}
{:role "reason" :content {:text "r1" :delta-reasoning? true}}
{:role "assistant" :content "A1"}
{:role "user" :content "Q2"}
{:role "reason" :content {:text "r2"}}
{:role "assistant" :content "A2"}]
:off))))

(testing "No user message - reasoningHistory \"turn\" leaves list unchanged"
(let [msgs [{:role "assistant" :content "A"}
{:role "reason" :content {:text "r"}}]]
(is (= msgs (#'llm-providers.openai-chat/prune-history msgs))))))
(is (= msgs (#'llm-providers.openai-chat/prune-history msgs :turn)))))

(testing "No user message - reasoningHistory \"off\" removes reason"
(is (match?
[{:role "assistant" :content "A"}]
(#'llm-providers.openai-chat/prune-history
[{:role "assistant" :content "A"}
{:role "reason" :content {:text "r"}}]
:off)))))

(deftest valid-message-test
(testing "Tool messages are always kept"
Expand Down
Loading