Skip to content

Comments

SearchScraper#31

Merged
PeriniM merged 11 commits intomainfrom
pre/beta
Feb 3, 2025
Merged

SearchScraper#31
PeriniM merged 11 commits intomainfrom
pre/beta

Conversation

@PeriniM
Copy link
Contributor

@PeriniM PeriniM commented Feb 3, 2025

🚀 Major API Update: SearchScraper Addition & SmartScraper Enhancement

🔄 Changes

✨ New Features

  • Added SearchScraper endpoint (🔍) for AI-powered web searches with reference URLs
  • Enhanced SmartScraper (🤖) to handle both webpage URLs and HTML content

🔀 Consolidation

  • Merged LocalScraper functionality into SmartScraper
  • SmartScraper now accepts either website_url or website_html parameter

📝 Documentation

  • Updated API reference and examples
  • Added new schema examples for structured search results
  • Updated READMEs with new capabilities

🎯 Benefits

  • Simplified API surface with fewer endpoints
  • More flexible content extraction options
  • New search capabilities with source attribution
  • Consistent AI-powered processing across all features

🧪 Testing

  • Added comprehensive tests for new SearchScraper endpoint
  • Updated SmartScraper tests to cover HTML input
  • All existing tests passing ✅

semantic-release-bot and others added 11 commits January 8, 2025 12:01
## [1.9.0-beta.6](v1.9.0-beta.5...v1.9.0-beta.6) (2025-01-08)

### Features

* add time varying timeout ([945b876](945b876))
* update doc readme ([c02c411](c02c411))

### Bug Fixes

* houses examples and typos ([c596c44](c596c44))
* make timeout optional ([49b8e4b](49b8e4b))

### chore

* fix _make_request not using it ([701a4c1](701a4c1))
* fix pyproject version ([3567034](3567034))

### Docs

* added api reference ([6929a7a](6929a7a))
* added api reference ([7b88876](7b88876))
* added cookbook reference ([e68c1bd](e68c1bd))
* added langchain-scrapegraph examples ([479dbdb](479dbdb))
* added open in colab badge ([c2fc1ef](c2fc1ef))
* added two langchain-scrapegraph examples ([8f3a87e](8f3a87e))
* added two new examples ([5fa2b42](5fa2b42))
* **cookbook:** added two new examples ([f67769e](f67769e))
* added wired langgraph react agent ([9f1e0cf](9f1e0cf))
* added zillow example ([7fad92c](7fad92c))
* api reference ([855c2e5](855c2e5))
* fixed cookbook images and urls ([f860167](f860167))
* github trending sdk ([320de37](320de37))
* link typo ([e1bfd6a](e1bfd6a))
* llama-index @VinciGit00 ([6de5eb2](6de5eb2))
* research agent ([6e06afa](6e06afa))
* updated new documentation urls ([1d0cb46](1d0cb46))
## [1.9.0-beta.7](v1.9.0-beta.6...v1.9.0-beta.7) (2025-02-03)

### Features

* add optional headers to request ([bb851d7](bb851d7))
* merged localscraper into smartscraper ([503dbd1](503dbd1))
* modified icons ([bcb9b0b](bcb9b0b))
* searchscraper ([2e04e5a](2e04e5a))
* updated readmes ([bfdbea0](bfdbea0))

### chore

* refactor examples ([8e00846](8e00846))
* **tests:** updated tests ([9149ce8](9149ce8))
@github-actions
Copy link

github-actions bot commented Feb 3, 2025

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails

Scanned Files

@PeriniM PeriniM merged commit f32d182 into main Feb 3, 2025
5 checks passed
@github-actions
Copy link

github-actions bot commented Feb 3, 2025

🎉 This PR is included in version 1.11.0-beta.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions
Copy link

github-actions bot commented Feb 3, 2025

🎉 This PR is included in version 1.11.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants