🤖 feat: add web_fetch tool for fetching web pages as markdown #742
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ammar-agent
force-pushed
the
web-fetch-tool-plan
branch
2 times, most recently
from
cdc7475 to
0279398
Compare
This comment has been minimized.
This comment has been minimized.
ammar-agent
force-pushed
the
web-fetch-tool-plan
branch
8 times, most recently
from
ff158ee to
509aed6
Compare
Closed
ammar-agent
force-pushed
the
web-fetch-tool-plan
branch
from
509aed6 to
381673a
Compare
Adds a new web_fetch tool that: - Fetches web pages using curl via the Runtime (respects workspace network context) - Extracts main content using Mozilla Readability - Converts to clean markdown using Turndown - Supports both markdown and plain text output formats Features: - Network isolation: requests originate from workspace, not Mux host - Curl handles redirects, SSL, encoding, compression natively - Output truncated to 64KB, HTML input limited to 5MB - Graceful error handling for DNS failures, timeouts, empty responses Dependencies added: - @mozilla/readability: article extraction - jsdom: DOM parsing for Readability - turndown: HTML to markdown conversion Also updates AGENTS.md testing section to clarify preferred test types: 1. True integration tests (no mocks) 2. Unit tests on pure/isolated logic _Generated with `mux`_
ammar-agent
force-pushed
the
web-fetch-tool-plan
branch
from
381673a to
093de8a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a new
web_fetchtool that:Features
Dependencies added
@mozilla/readability: article extractionjsdom: DOM parsing for Readabilityturndown: HTML to markdown conversionTesting
9 integration tests using real runtime (no mocks):
Also updates AGENTS.md testing section to clarify preferred test types:
Generated with
mux