Update the version constrains of google-cloud-storage
#2079
Conversation
Signed-off-by: Yu Ishikawa <yu-iskw@users.noreply.github.com>
📝 WalkthroughWalkthroughThe Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes Poem
Pre-merge checks✅ Passed checks (3 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
👋 @yu-iskw |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Fix all issues with AI Agents 🤖
In @pyproject.toml:
- Line 25: The google-cloud-storage version range in pyproject.toml is too
permissive (>=2.4,<4.0) and must be tightened to <3.2 for compatibility with
dbt-bigquery 1.10.3; update the dependency constraint for google-cloud-storage
to use an upper bound of <3.2 in pyproject.toml and run dependency
resolution/tests to ensure nothing in the code (especially any uses of
bucket/blob APIs that rely on checksum behavior, num_retries, text_mode, or
retry semantics) breaks with google-cloud-storage 3.x before accepting any
broader range.
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
pyproject.toml
| posthog = "<3.0.0" | ||
| boto3 = "<2.0.0" | ||
| google-cloud-storage = "<3.0.0" | ||
| google-cloud-storage = ">=2.4,<4.0" |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check latest google-cloud-storage version and any 4.x releases
echo "=== google-cloud-storage releases (3.x and 4.x) ==="
curl -s https://pypi.org/pypi/google-cloud-storage/json | jq -r '.releases | keys[]' | grep -E '^(3\.|4\.)' | sort -V | tail -20
echo -e "\n=== dbt-bigquery 1.10.3 requirements ==="
curl -s https://pypi.org/pypi/dbt-bigquery/1.10.3/json | jq -r '.info.requires_dist[]? | select(contains("google-cloud-storage"))' 2>/dev/null || echo "Could not fetch dbt-bigquery info"
echo -e "\n=== Current google-cloud-storage latest version ==="
curl -s https://pypi.org/pypi/google-cloud-storage/json | jq -r '.info.version'Repository: elementary-data/elementary
Length of output: 328
🌐 Web query:
google-cloud-storage 3.0 breaking changes migration python
💡 Result:
Key breaking changes in google-cloud-storage 3.0 (Python) and how to migrate:
- Default checksum behavior changed
- Upload default checksum changed from None -> "auto" (checksums computed/validated automatically).
- Download default checksum changed from "md5" -> "auto".
- Action: explicitly set checksum if you relied on the old behavior (e.g., blob.upload_from_file(..., checksum="md5"/None) or blob.download_as_bytes(..., checksum="md5"/None)) and test uploads/downloads. [1][2]
- Retry/argument changes
- Deprecated positional argument num_retries removed; use the retry parameter (named) with google.api_core.retry.Retry or library retry helpers instead. Replace calls like blob.download_to_filename("f", 3) with blob.download_to_filename("f", retry=my_retry). [1][2]
- text_mode removed
- Any API that accepted text_mode has that argument removed; pass/encode bytes instead or handle text decoding yourself before/after calls. [1][2]
- Media retry behavior
- Media operation retries now behave the same as other retries (this may change effective retry semantics/defaults). Action: review and, if needed, supply explicit retry policies for media operations. [1]
- Blob.download_to_filename() behavior change
- On 404, the client now deletes an empty destination file created before the download. Action: if your code relied on an empty file remaining on 404, adjust logic to check for existence or catch the NotFound error. [1]
Migration checklist
- Search/replace positional num_retries usages → named retry parameter.
- Audit places using text_mode and remove/replace with explicit encoding/decoding.
- Set explicit checksum options where your code depends on MD5/no-checksum.
- Review retry policies for media operations and tests for download-on-404 behavior.
- Run integration tests (uploads/downloads, error paths) after upgrade. [1][2][3]
Sources:
- Google Cloud Storage release notes / 3.0.0 changelog. [1]
- google-cloud-storage changelog / docs page. [2]
- 3.0.0 client reference / README (migration notes). [3]
Fix the upper bound to <3.2 for compatibility with dbt-bigquery 1.10.3.
The current constraint >=2.4,<4.0 is incompatible with dbt-bigquery 1.10.3, which requires google-cloud-storage<3.2,>=2.4. The upper bound should be <3.2 instead of <4.0. Versions 3.2 through 3.7.0 exist and will break dbt-bigquery 1.10.3 compatibility.
Additionally, google-cloud-storage 3.0 introduced several breaking changes (checksum behavior, removed num_retries parameter, removed text_mode, modified retry semantics) that require testing to ensure the codebase is compatible.
🤖 Prompt for AI Agents
In @pyproject.toml around line 25, The google-cloud-storage version range in
pyproject.toml is too permissive (>=2.4,<4.0) and must be tightened to <3.2 for
compatibility with dbt-bigquery 1.10.3; update the dependency constraint for
google-cloud-storage to use an upper bound of <3.2 in pyproject.toml and run
dependency resolution/tests to ensure nothing in the code (especially any uses
of bucket/blob APIs that rely on checksum behavior, num_retries, text_mode, or
retry semantics) breaks with google-cloud-storage 3.x before accepting any
broader range.
1 participant
I encounter the conflicts between dbt-bigquery and elementary-data. So, I want to loosen the version constrains of
google-cloud-storage.https://github.com/dbt-labs/dbt-adapters/blob/1e7448f9b6c3440d28fefd5b34b6dcf86c36def1/dbt-bigquery/pyproject.toml#L30
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.