[Java] Allow Arrow API to read file contents from inputStream instead of only Path

[swapnil-bidgely]

Describe the enhancement requested

Currently, only uri is allowed as interface if we wish to read any file format

public FileSystemDatasetFactory(BufferAllocator allocator, NativeMemoryPool memoryPool, FileFormat format, String uri) {
        super(allocator, memoryPool, createNative(format, uri));
    }

I have a usecase where in I would want to pass an inputstream and based on it I wished to read and convert to the target fileformat.
Could someone from the community help me with this requirement, as this would help me go away from ParquetReader library or even AvroReader library and use Arrow instead

Component(s)

Java

[swapnil-bidgely]

@kou Can you pls route this to appropriate personnel for clarification ?

[kou]

This issue is for the Java implementation, right?

@vibhatha @lidavidm Could you take a look at this?

[vibhatha]

On top of my head what I could suggest as a workaround is probably write a util. Let me check this. Need some time though.

[lidavidm]

Because this involves the JNI binding, we would have to somehow pass a file descriptor over to the C++ API, or otherwise bridge a Java InputStream into an Arrow stream. Also, in general, Dataset isn't made for that kind of use case: note that even the Python bindings only accept URIs, not file objects: https://arrow.apache.org/docs/python/generated/pyarrow.dataset.dataset.html

[assignUser]

@lidavidm comment sounds like this wont' happen for technical reasons.