don't show warning about bucket

[djouallah]

Apache Iceberg version

main (development)

Please describe the bug 🐞

using 0.9 rc1

when connection to polaris, I am getting this warning

WARNING:pyiceberg.io.pyarrow:PyArrow FileIO overriding S3 bucket region for bucket xxxxxx: provided region None, actual region us-east-1

the system is exposing information that I don't expect the users to know anything about it ?

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[omkenge]

ref this #1706

[kevinjqliu]

Thanks for raising this issue! I also found an issue where resolve_s3_region is called too many times (#1708). I think we should fix both for the 0.9.0 release.

@omkenge thanks for the PR! I think we need a way to resolve #1708 too

[omkenge]

What do you think ? @kevinjqliu
By introducing a configuration flag, we allow users (or integration tests) to disable automatic S3 region resolution.
Updated #1706

[Fokko]

Thanks @djouallah for reporting this. I think we should suppress the warning when Provided Region is None. There is zero value in that. The message is just to avoid that you start pulling large amounts of data across regions without being aware of it.

By introducing a configuration flag, we allow users (or integration tests) to disable automatic S3 region resolution.

As mentioned in #1708 I think we can also explore other options. An important part of testing is that we test the code-paths that are used by the users. Therefore I'm not a fan of disabling this for the integration tests.

[kevinjqliu]

Closing the loop here, in #1709 we change the behavior to not emit warnings when s3.region is not provided.

We will only emit a warning when s3.resolve-region is set to True and the provided s3.region is different from the actual bucket region.

Detailed behavior here #1709 (review)

Thanks again @djouallah

[djouallah]

thank you guys, pyiceberg 0.9 is a great release already !!!

[kevinjqliu]

Thanks @djouallah The new 0.9.0 RC2 is out (https://pypi.org/project/pyiceberg/0.9.0rc2/)
Let us know if you run into any issue :)

You can also verify the release on the devlist thread