Skip to content

Conversation

@gweaverbiodev
Copy link
Contributor

@gweaverbiodev gweaverbiodev commented Aug 12, 2025

Rationale for this change

Currently, it is not possible to configure anonymous in the underlying s3fs client (False by default).

Resolves #2126

Are these changes tested?

Yes, unit tests added to confirm the setting is used when the property is set.

Are there any user-facing changes?

Yes, users can now set the property s3.anonymous to True to enable. If not set, the behavior will be the same as it currently is.

@gweaverbiodev gweaverbiodev marked this pull request as ready for review August 12, 2025 17:18
Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gmweaver for adding this 🙌

Probably we want to add this to the docs as well: https://github.com/apache/iceberg-python/blob/main/mkdocs/docs/configuration.md#s3

Do you know if there is a PyArrow equialent? If so, it would be nice to add that option as well.

@gweaverbiodev
Copy link
Contributor Author

gweaverbiodev commented Aug 12, 2025

Do you know if there is a PyArrow equialent? If so, it would be nice to add that option as well.

pyarrow has a similar config updated the PR to include.

@gweaverbiodev gweaverbiodev changed the title add support to configure anonymous in s3 client add support to configure anonymous in s3 clients Aug 12, 2025
@Fokko
Copy link
Contributor

Fokko commented Aug 13, 2025

@gmweaver Thanks for being on top of this, could you run make lint to fix the style issues?

@gweaverbiodev
Copy link
Contributor Author

@Fokko sorry missed the make lint in the docs, updated

Copy link
Contributor

@kevinjqliu kevinjqliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kevinjqliu kevinjqliu merged commit a7f6c08 into apache:main Aug 15, 2025
11 checks passed
@kevinjqliu
Copy link
Contributor

Thanks for the PR @gmweaver and thank you @Fokko for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add s3.anon as configurable property for S3 FileIO

3 participants