Skip to content

Conversation

@codegen-sh
Copy link
Contributor

@codegen-sh codegen-sh bot commented Mar 20, 2025

This PR implements a new BrowseWeb tool that allows Codegen to browse the web, similar to the LinearViewIssue tool.

Changes:

  • Added a new web module in src/codegen/extensions/tools/web/
  • Implemented two main tools:
    • web_search_tool: Searches the web and returns content snippets from search results
    • web_view_page_tool: Views the content of a specific webpage
  • Added necessary dependencies:
    • beautifulsoup4 for HTML parsing
    • responses for testing
  • Added unit tests for the new tools

Implementation Details:

  • The web search tool currently returns mock data, but is designed to be easily extended to use a real search API
  • The web view page tool uses requests and BeautifulSoup to fetch and parse web pages
  • Both tools follow the same pattern as other tools in the codebase, returning Observation objects

Next Steps:

  • Integrate with a real search API (Google Custom Search, Bing Search, etc.)
  • Add more advanced web scraping capabilities
  • Improve error handling and rate limiting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants