feat: extra text block #4189
Open
feat: extra text block #4189
+182
−70
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - 我发现了 3 个问题，并给出了一些总体反馈：
assemble_context中的向后兼容降级逻辑在ProviderRequest与各个 provider source 之间不一致：OpenAI / Gemini / Anthropic 只要存在单一文本块就会降级为纯文本字符串（即使该块来自extra_content_blocks），而ProviderRequest.assemble_context只会在不存在额外块或图片时才降级；建议统一这些条件，以便在多内容块场景下，各个 provider 的行为保持一致、更可预测。- 在
process_llm_request中构建system_content时，system_parts使用"".join(...)拼接，这会生成一个没有任何分隔符的长串（例如User ID...Nickname...Group name...Current datetime...），影响可读性；建议使用换行或其他明显分隔符（如"\n".join(system_parts)）来拼接，以符合预期的可读格式。
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The backward‑compatibility downgrade logic in `assemble_context` is inconsistent between `ProviderRequest` and the provider sources: OpenAI/Gemini/Anthropic downgrade to a plain text string whenever there is a single text block (even if it came from `extra_content_blocks`), whereas `ProviderRequest.assemble_context` only downgrades when there are no extra blocks or images; consider aligning these conditions so that multi‑block usage behaves predictably across providers.
- When building `system_content` in `process_llm_request`, `system_parts` are concatenated with `"".join(...)`, which will produce a single run‑on string (e.g., `User ID...Nickname...Group name...Current datetime...`) without separators; consider joining with newlines or a clear delimiter (e.g., `"\n".join(system_parts)`) to match the intended readable format.
## Individual Comments
### Comment 1
<location> `packages/astrbot/process_llm_request.py:243-244` </location>
<code_context>
+ req.extra_content_blocks.append({"type": "text", "text": quoted_text})
+
+ # 统一包裹所有系统提醒
+ if system_parts:
+ system_content = (
+ "<system_reminder>" + "".join(system_parts) + "</system_reminder>"
+ )
</code_context>
<issue_to_address>
**issue (bug_risk):** System reminder pieces are concatenated without separators, which harms readability and may change semantics.
Previously these system details were separated by newlines, but `"".join(system_parts)` now produces `<system_reminder>User ID: ...Group name: ...Current datetime: ...</system_reminder>` with no delimiters. This reduces readability and may break existing prompts that depend on line breaks. Consider joining with newlines (e.g. `"\n".join(system_parts)` or prefixing each part with `"\n"`) to preserve the prior structure.
</issue_to_address>
### Comment 2
<location> `astrbot/core/provider/sources/openai_source.py:665-667` </location>
<code_context>
- return user_content
- return {"role": "user", "content": text}
+
+ # 如果只有文本且没有额外内容块，返回简单格式以保持向后兼容
+ if len(content_blocks) == 1 and content_blocks[0]["type"] == "text":
+ return {"role": "user", "content": content_blocks[0]["text"]}
+
</code_context>
<issue_to_address>
**suggestion (bug_risk):** The downgrade-to-plain-text condition ignores `extra_content_blocks` and may misclassify messages.
The comment says “只有文本且没有额外内容块”, but the code only checks `len(content_blocks) == 1` and `type == "text"`. As a result:
- A message with empty `text` and a single text `extra_content_block` (no images) will be downgraded to `{"content": <extra_text>}`, losing the original block structure.
- This differs from `ProviderRequest.assemble_context`, which only downgrades when `not self.extra_content_blocks and not self.image_urls`.
To keep block structure when content comes from `extra_content_blocks` or multimodal input, consider applying the same guard (also ensuring there are no images and no extra blocks beyond the main prompt text).
```suggestion
# 如果只有主文本且没有额外内容块和图片，返回简单格式以保持向后兼容
# 注意：这里与 ProviderRequest.assemble_context 保持一致，
# 仅在没有 extra_content_blocks 且没有 image_urls 时才降级为纯文本，
# 避免把来自 extra_content_blocks 的内容或多模态消息误判为简单文本消息。
if (
not extra_content_blocks
and not image_urls
and len(content_blocks) == 1
and content_blocks[0]["type"] == "text"]
):
return {"role": "user", "content": content_blocks[0]["text"]}
```
</issue_to_address>
### Comment 3
<location> `astrbot/core/provider/sources/gemini_source.py:842-843` </location>
<code_context>
- return user_content
- return {"role": "user", "content": text}
+
+ # 如果只有文本且没有额外内容块，返回简单格式以保持向后兼容
+ if len(content_blocks) == 1 and content_blocks[0]["type"] == "text":
+ return {"role": "user", "content": content_blocks[0]["text"]}
+
</code_context>
<issue_to_address>
**suggestion:** Same downgrade condition issue as OpenAI: extra-only content can be collapsed to plain text unexpectedly.
This uses the same overly-broad downgrade rule as `openai_source.assemble_context`: any single text `content_block` is converted to `{"content": <text>}`, even when it comes solely from `extra_content_blocks`. In cases where callers only pass an extra block (e.g., system reminder, quoted message) and no prompt/images, this collapses the structure and loses semantics. Please tighten the condition (as in `ProviderRequest.assemble_context`) so that only a plain user prompt with no extras/images is downgraded.
Suggested implementation:
```python
# 如果只有文本且没有图片或额外内容块，返回简单格式以保持向后兼容
# 注意：仅在调用方未提供 extra_content_blocks 和 image_urls 时才降级，
# 避免“只传额外块（例如系统提醒、引述消息）”的场景被错误折叠为纯文本。
if (
len(content_blocks) == 1
and content_blocks[0]["type"] == "text"
and not image_urls
and not extra_content_blocks
):
return {"role": "user", "content": content_blocks[0]["text"]}
```
1. 确认 `assemble_context` 的调用方在仅传「额外内容块」而不传主 `text` 时，依然会把这些块放入 `extra_content_blocks`，而不是混入主文本块中；否则需要对构造 `content_blocks` 的逻辑做类似 `ProviderRequest.assemble_context` 的拆分（例如区分「主 user 提示」与「extra blocks」的来源）。
2. 建议对 `ProviderRequest.assemble_context` 当前的降级条件进行对照，确保这里与那里的降级语义保持一致（都仅在“单一主文本、无图片、无 extra 块”时返回简化结构）。
</issue_to_address>请帮我变得更有用！欢迎在每条评论上点 👍 或 👎，我会根据你的反馈不断改进评审质量。
Original comment in English
Hey - I've found 3 issues, and left some high level feedback:
- The backward‑compatibility downgrade logic in
assemble_contextis inconsistent betweenProviderRequestand the provider sources: OpenAI/Gemini/Anthropic downgrade to a plain text string whenever there is a single text block (even if it came fromextra_content_blocks), whereasProviderRequest.assemble_contextonly downgrades when there are no extra blocks or images; consider aligning these conditions so that multi‑block usage behaves predictably across providers. - When building
system_contentinprocess_llm_request,system_partsare concatenated with"".join(...), which will produce a single run‑on string (e.g.,User ID...Nickname...Group name...Current datetime...) without separators; consider joining with newlines or a clear delimiter (e.g.,"\n".join(system_parts)) to match the intended readable format.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The backward‑compatibility downgrade logic in `assemble_context` is inconsistent between `ProviderRequest` and the provider sources: OpenAI/Gemini/Anthropic downgrade to a plain text string whenever there is a single text block (even if it came from `extra_content_blocks`), whereas `ProviderRequest.assemble_context` only downgrades when there are no extra blocks or images; consider aligning these conditions so that multi‑block usage behaves predictably across providers.
- When building `system_content` in `process_llm_request`, `system_parts` are concatenated with `"".join(...)`, which will produce a single run‑on string (e.g., `User ID...Nickname...Group name...Current datetime...`) without separators; consider joining with newlines or a clear delimiter (e.g., `"\n".join(system_parts)`) to match the intended readable format.
## Individual Comments
### Comment 1
<location> `packages/astrbot/process_llm_request.py:243-244` </location>
<code_context>
+ req.extra_content_blocks.append({"type": "text", "text": quoted_text})
+
+ # 统一包裹所有系统提醒
+ if system_parts:
+ system_content = (
+ "<system_reminder>" + "".join(system_parts) + "</system_reminder>"
+ )
</code_context>
<issue_to_address>
**issue (bug_risk):** System reminder pieces are concatenated without separators, which harms readability and may change semantics.
Previously these system details were separated by newlines, but `"".join(system_parts)` now produces `<system_reminder>User ID: ...Group name: ...Current datetime: ...</system_reminder>` with no delimiters. This reduces readability and may break existing prompts that depend on line breaks. Consider joining with newlines (e.g. `"\n".join(system_parts)` or prefixing each part with `"\n"`) to preserve the prior structure.
</issue_to_address>
### Comment 2
<location> `astrbot/core/provider/sources/openai_source.py:665-667` </location>
<code_context>
- return user_content
- return {"role": "user", "content": text}
+
+ # 如果只有文本且没有额外内容块，返回简单格式以保持向后兼容
+ if len(content_blocks) == 1 and content_blocks[0]["type"] == "text":
+ return {"role": "user", "content": content_blocks[0]["text"]}
+
</code_context>
<issue_to_address>
**suggestion (bug_risk):** The downgrade-to-plain-text condition ignores `extra_content_blocks` and may misclassify messages.
The comment says “只有文本且没有额外内容块”, but the code only checks `len(content_blocks) == 1` and `type == "text"`. As a result:
- A message with empty `text` and a single text `extra_content_block` (no images) will be downgraded to `{"content": <extra_text>}`, losing the original block structure.
- This differs from `ProviderRequest.assemble_context`, which only downgrades when `not self.extra_content_blocks and not self.image_urls`.
To keep block structure when content comes from `extra_content_blocks` or multimodal input, consider applying the same guard (also ensuring there are no images and no extra blocks beyond the main prompt text).
```suggestion
# 如果只有主文本且没有额外内容块和图片，返回简单格式以保持向后兼容
# 注意：这里与 ProviderRequest.assemble_context 保持一致，
# 仅在没有 extra_content_blocks 且没有 image_urls 时才降级为纯文本，
# 避免把来自 extra_content_blocks 的内容或多模态消息误判为简单文本消息。
if (
not extra_content_blocks
and not image_urls
and len(content_blocks) == 1
and content_blocks[0]["type"] == "text"
):
return {"role": "user", "content": content_blocks[0]["text"]}
```
</issue_to_address>
### Comment 3
<location> `astrbot/core/provider/sources/gemini_source.py:842-843` </location>
<code_context>
- return user_content
- return {"role": "user", "content": text}
+
+ # 如果只有文本且没有额外内容块，返回简单格式以保持向后兼容
+ if len(content_blocks) == 1 and content_blocks[0]["type"] == "text":
+ return {"role": "user", "content": content_blocks[0]["text"]}
+
</code_context>
<issue_to_address>
**suggestion:** Same downgrade condition issue as OpenAI: extra-only content can be collapsed to plain text unexpectedly.
This uses the same overly-broad downgrade rule as `openai_source.assemble_context`: any single text `content_block` is converted to `{"content": <text>}`, even when it comes solely from `extra_content_blocks`. In cases where callers only pass an extra block (e.g., system reminder, quoted message) and no prompt/images, this collapses the structure and loses semantics. Please tighten the condition (as in `ProviderRequest.assemble_context`) so that only a plain user prompt with no extras/images is downgraded.
Suggested implementation:
```python
# 如果只有文本且没有图片或额外内容块，返回简单格式以保持向后兼容
# 注意：仅在调用方未提供 extra_content_blocks 和 image_urls 时才降级，
# 避免“只传额外块（例如系统提醒、引述消息）”的场景被错误折叠为纯文本。
if (
len(content_blocks) == 1
and content_blocks[0]["type"] == "text"
and not image_urls
and not extra_content_blocks
):
return {"role": "user", "content": content_blocks[0]["text"]}
```
1. 确认 `assemble_context` 的调用方在仅传「额外内容块」而不传主 `text` 时，依然会把这些块放入 `extra_content_blocks`，而不是混入主文本块中；否则需要对构造 `content_blocks` 的逻辑做类似 `ProviderRequest.assemble_context` 的拆分（例如区分「主 user 提示」与「extra blocks」的来源）。
2. 建议对 `ProviderRequest.assemble_context` 当前的降级条件进行对照，确保这里与那里的降级语义保持一致（都仅在“单一主文本、无图片、无 extra 块”时返回简化结构）。
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Contributor
Author
|
这个PR将是其他模块优化的基石 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🎯 动机 / Motivation
解决的问题：
添加的功能：
📝 改动点 / Modifications
核心文件修改：
core/provider/entities.pycore/provider/sources/openai_source.pycore/provider/sources/gemini_source.pycore/provider/sources/anthropic_source.pypackages/astrbot/process_llm_request.py实现的功能：
🖼️ 测试结果 / Test Results
测试场景：
测试输出示例：
✅ 检查清单 / Checklist
🚀 使用示例
需要注意的是，本次PR并未对所以 本人从未使用的功能进行优化。
比如知识库。
✦ 这个 PR 为 AstrBot 的多模态消息处理奠定了基础，同时保持了完全的向后兼容性！🎯
Summary by Sourcery
在保持现有单文本行为向后兼容的前提下，为用户消息添加对多个内容块的结构化支持。
新功能：
ProviderRequest上引入extra_content_blocks字段，用于携带额外的用户消息片段，例如系统提醒、图像描述和引用消息。增强点：
<system_reminder>文本块，而不是直接注入到提示词中。<image_caption>、<Quoted Message>）追加在用户消息之后，而非前置到提示词前面。Original summary in English
Summary by Sourcery
Add structured support for multiple content blocks in user messages while keeping existing single-text behavior backward compatible.
New Features:
Enhancements: