[WIP] Redesign Android AAR for modular backend architecture

[Copilot]

Task: Design Analysis for Modular Backend Architecture in ExecutorTorch Android AAR

Plan:

• Explore current Android AAR structure and build system
• Understand current backend integration (XNNPACK, QNN, Vulkan)
• Create comprehensive design documentation covering:
  • Technical feasibility analysis
  • Architecture design recommendations
  • Implementation challenges and solutions
  • Build system design
  • Performance implications
  • Developer experience API design
  • Migration strategy
  • Trade-offs analysis
  • Alternative approaches
  • Implementation roadmap
  • Risk assessment
  • Code examples

Current findings:

• Core AAR is built from extension/android/ with CMake + Gradle
• Backends (XNNPACK, QNN, Vulkan) are statically linked into single libexecutorch.so
• Build script build_android_library.sh handles compilation for multiple ABIs
• Java Module class loads library via NativeLoader.loadLibrary("executorch")

Original prompt

Task: Design Analysis for Modular Backend Architecture in ExecutorTorch Android AAR

Context

Currently, ExecutorTorch's Android AAR packaging has the following structure:

• Default AAR includes XNNPACK backend (statically compiled into the .so)
• Optional separate AARs for QNN and Vulkan backends (also statically compiled)
• Apps must choose which AAR to use at compile time

This creates inflexibility - apps cannot dynamically select backends at runtime based on device capabilities.

Proposed Solution

Redesign the AAR architecture to separate backends into independent loadable modules:

Core AAR:

• Contains base ExecutorTorch runtime
• Includes XNNPACK as default lightweight backend
• Exports stable C API for backend registration

Backend Plugin AARs (optional dependencies):

• executorch-backend-qnn.aar - QNN backend in separate .so
• executorch-backend-vulkan.aar - Vulkan backend in separate .so
• Apps can include 0, 1, or multiple backend AARs
• Runtime loads backends via System.loadLibrary() on demand

Research Requirements

Please analyze and provide detailed recommendations on:

1. Technical Feasibility

• How to design a stable C/C++ ABI for backend plugins
• Symbol export/import mechanisms between core and backend .so files
• JNI loading order and initialization sequence
• Backend registration and discovery patterns
• Handle dependencies (e.g., QNN requires libQnnHtp.so, libQnnCpu.so)

2. Architecture Design

Compare these approaches:

• Plugin Registry Pattern: Backends register themselves via exported functions
• Pure Dynamic Loading: Use dlopen/dlsym to load backends at runtime
• Hybrid Approach: Compile-time optional linking with runtime fallback

Recommend the best approach with rationale.

3. Implementation Challenges

Identify and propose solutions for:

• ABI compatibility across versions (core vs backends)
• Symbol conflicts when multiple backends loaded
• Dependency management in Gradle/Maven
• Error handling when backends fail to load
• Version compatibility matrix
• Debugging across multiple .so files

4. Build System Design

• CMake configuration for core + plugin builds
• Android AAR packaging strategy
• Gradle dependency declarations (required vs optional)
• How to handle different ABIs (arm64-v8a, armeabi-v7a, x86_64)

5. Performance Implications

• Overhead of virtual function dispatch vs static linking
• Memory overhead of multiple loaded .so files
• Impact on app startup time
• Any optimization strategies

6. Developer Experience

Design the API surface:

Java/Kotlin layer:

// How should apps use this?
dependencies {
    implementation 'org.pytorch:executorch-core:1.0.0'
    implementation 'org.pytorch:executorch-backend-qnn:1.0.0' // optional
}

// Runtime usage - what should this look like?
val module = Module.load(modelPath, backend = "qnn")

7. Migration Strategy
	∙	How to maintain backward compatibility with current static linking approach?
	∙	Phased rollout plan
	∙	Testing strategy for the new architecture
8. Trade-offs Analysis
Provide a clear comparison:
Benefits:
	∙	Runtime backend selection
	∙	Smaller base APK size
	∙	Independent backend updates
	∙	Better device optimization
Costs:
	∙	Implementation complexity
	∙	ABI maintenance burden
	∙	Potential for runtime errors
	∙	Debugging complexity
9. Alternative Approaches
Are there other solutions worth considering?
	∙	App bundles with dynamic feature modules?
	∙	Compile-time code generation?
	∙	Keeping current approach but improving it?
Deliverable
Please provide:
	1.	Recommended architecture with detailed design
	2.	API specifications (Java + C++ interfaces)
	3.	Implementation roadmap with phases and estimates
	4.	Risk assessment with mitigation strategies
	5.	Code examples showing the proposed usage patterns
	6.	Decision matrix comparing different approaches

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17062

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Cancelled Job

As of commit 1816658 with merge base cfc71fe ():

CANCELLED JOB - The following job was cancelled. Please retry:

• Running Copilot coding agent / copilot (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.