Trying to use for Android but keep getting No DefaultAllocationManager found on classpath

[kpeck-north]

Describe the bug, including details regarding any error messages, version, and platform.

I have all the dependencies in place, code compiles just fine but when I run it and attempt to parse from an arrow stream like this (or using any of the parsers like from a file)

val df = DataFrame.readArrowFeather(channel.toInputStream(null))

I always get

Caused by: java.lang.RuntimeException: No DefaultAllocationManager found on classpath. Can't allocate Arrow buffers. Please consider adding arrow-memory-netty or arrow-memory-unsafe as a dependency.

Same for
val rootAllocator = RootAllocator()
val fileInputStream = FileInputStream(File(filepath))
val arrowReader = ArrowStreamReader(fileInputStream, rootAllocator)

Asking Gradle for my arrow dependencies I see

| +--- org.jetbrains.kotlinx:dataframe-arrow:1.0.0-dev-7570
| | +--- org.apache.arrow:arrow-vector:18.1.0
| | | +--- org.apache.arrow:arrow-format:18.1.0
| | | +--- org.apache.arrow:arrow-memory-core:18.1.0
| | +--- org.apache.arrow:arrow-format:18.1.0 ()
| | +--- org.apache.arrow:arrow-memory-unsafe:18.1.0
| | | --- org.apache.arrow:arrow-memory-core:18.1.0 ()
+--- org.jetbrains.kotlinx:dataframe-arrow:1.0.0-dev-7570 ()
+--- org.apache.arrow:arrow-memory-unsafe:18.1.0 ()

This happens even if I don't use org.jetbrains.kotlinx.dataframe and just call the Apache Arrow parser directly as seen in second code example above.

I have tried adding both arrow-memory-unsage and arrow-memory-netty with the same result. I have been beating my head against a wall for the last few days. Is Android supported with this library? The data from the server comes to me in arrow-stream format so I need to be able to parse it to shows charts.

I have been able to parse the data just fine with Python and the web team is parsing it as well. I need to do this for Native Android and Native iOS.

Component(s)

Integration

[raulcd]

Thanks for opening the issue. As this seems related to the Arrow Java implementation, I've transferred the issue to the arrow-java repository.

[lidavidm]

I don't think we explicitly support or test Android. It seems ClassLoader#getSystemResources must behave differently. There is a system property you can try setting to force it to pick a particular implementation:

arrow-java/memory/memory-core/src/main/java/org/apache/arrow/memory/DefaultAllocationManagerOption.java

Lines 56 to 98 in 7618274

	@SuppressWarnings("nullness:argument") // enum types valueOf are implicitly non-null
	@VisibleForTesting
	public static AllocationManagerType getDefaultAllocationManagerType() {
	AllocationManagerType ret = AllocationManagerType.Unknown;
	try {
	String envValue = System.getenv(ALLOCATION_MANAGER_TYPE_ENV_NAME);
	ret = AllocationManagerType.valueOf(envValue);
	} catch (IllegalArgumentException | NullPointerException e) {
	// ignore the exception, and make the allocation manager type remain unchanged
	}
	// system property takes precedence
	try {
	String propValue = System.getProperty(ALLOCATION_MANAGER_TYPE_PROPERTY_NAME);
	ret = AllocationManagerType.valueOf(propValue);
	} catch (IllegalArgumentException | NullPointerException e) {
	// ignore the exception, and make the allocation manager type remain unchanged
	}
	return ret;
	}
	static AllocationManager.Factory getDefaultAllocationManagerFactory() {
	if (DEFAULT_ALLOCATION_MANAGER_FACTORY != null) {
	return DEFAULT_ALLOCATION_MANAGER_FACTORY;
	}
	AllocationManagerType type = getDefaultAllocationManagerType();
	switch (type) {
	case Netty:
	DEFAULT_ALLOCATION_MANAGER_FACTORY = getNettyFactory();
	break;
	case Unsafe:
	DEFAULT_ALLOCATION_MANAGER_FACTORY = getUnsafeFactory();
	break;
	case Unknown:
	LOGGER.info("allocation manager type not specified, using netty as the default type");
	DEFAULT_ALLOCATION_MANAGER_FACTORY = getFactory(CheckAllocator.check());
	break;
	default:
	throw new IllegalStateException("Unknown allocation manager type: " + type);
	}
	return DEFAULT_ALLOCATION_MANAGER_FACTORY;
	}

[may040]

I have the same problem in Eclipse PDE in my bundle.
I even added the arrow-memory-netty-18.1.0.jar explicitly to the classpath, but I still keep getting this error.
Additionally, I added the following to the VM arguments in the run configuration:
--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED

But despite all of that, nothing seems to work.