[Feature Request] Support for NPU

[zeus-cht]

I am encountering a compatibility issue when running this project on NPU hardware (Huawei Ascend). The project currently depends on torch_memory_saver, which seems to be heavily optimized for CUDA memory management and fragmentation reduction.

Context

• Dependency: torch_memory_saver (likely introduced via sglang or vllm).
• Hardware: Huawei Ascend NPU (CANN).
• Current Behavior: torch_memory_saver specifically hooks into CUDA-related memory APIs (cudaMalloc, etc.). When running on NPU, it either fails to initializ

Question
Does the roadmap include support for non-CUDA backends like NPU?

[zheng-da]

We don't have it in our roadmap currently. You are welcome to contribute :)