S3Path.is_file() always returns True, causing iterdir() to raise NotADirectoryError — should check not is_dir() instead

[csniu]

Description

When using UPath with S3 (via s3fs/fsspec), S3Path.is_file() appears to always return True (even for “directory-like” prefixes). As a result, S3Path.iterdir() treats S3 prefixes as files and raises NotADirectoryError, making it impossible (or unreliable) to list “directories” on S3.

Given that object stores don’t have real directories, checking not is_dir() (instead of is_file()) is a more reliable guard for iterdir().

Steps to reproduce

from upath import UPath

p = UPath("s3://my-bucket/some/prefix/")

print("is_file:", p.is_file())   # True
print("is_dir:", p.is_dir())    # True
print("is_file:", p.is_file())   # False
print("is_dir:", p.is_dir())    # True

Environment

• upath version: 0.3.7
• fsspec version: 2025.12.0
• s3fs version: 2025.12.0
• Python: 3.12

[ap--]

Thank you for reporting @csniu

Could you write a standalone test in update/tests/implementations/test_s3.py that reproduces the is_file() behavior you show in your example code snippet?

The switch to not .is_dir() for iterdir already landed on main in #506 and will be available in 0.3.8

[csniu]

I investigated further and it looks like this is an s3fs behavior: when a “file key” and a “directory-like prefix” share the same name, is_file() can be reported as True initially.

For example, if I have these two objects:

• s3://my-bucket/some/prefix (an object whose key is exactly some/prefix)
• s3://my-bucket/some/prefix/file.txt (an object under the some/prefix/ prefix)

Then:

from upath import UPath

p = UPath("s3://my-bucket/some/prefix/")

print("is_file:", p.is_file())  # True
print("is_dir:", p.is_dir())    # True
print("is_file:", p.is_file())  # False
print("is_dir:", p.is_dir())    # True

So the is_file() returns True case seems to come from the key/prefix collision ambiguity in s3fs (and then a subsequent is_dir() call changes the cached classification).

[ap--]

Thanks for following up!

So I would recommend to report this using an example without UPath and only s3fs to the s3fs issue tracker.

In the general case S3FileSystem.isfile() and S3FileSystem.isdir() should be independent of each other.