Skip to content

[bugfix][NPU]:Fix bug that correctly obtains device type #1229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Feng0w0 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[bugfix][NPU]:Fix bug that correctly obtains device type #1229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion diffsynth/models/wan_video_dit.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def rope_apply(x, freqs, num_heads):
x = rearrange(x, "b s (n d) -> b s n d", n=num_heads)
x_out = torch.view_as_complex(x.to(torch.float64).reshape(
x.shape[0], x.shape[1], x.shape[2], -1, 2))
freqs = freqs.to(torch.complex64) if freqs.device == "npu" else freqs
freqs = freqs.to(torch.complex64) if freqs.device.type == "npu" else freqs
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While this change correctly fixes the device type check, there might be an underlying issue with the mixed-precision logic on NPU devices.

On line 94, x_out is created as a complex128 tensor (from torch.float64). With this change, freqs is converted to complex64 on NPU devices.

When x_out * freqs is executed on line 97, PyTorch's type promotion will likely cast freqs back to complex128 to match x_out's dtype. This could render the conversion to complex64 ineffective. If complex128 operations are unsupported on NPU, this could still lead to errors.

For this to work as intended (using complex64 on NPU), x_out should probably also be created as complex64. This would involve changing x.to(torch.float64) on line 94 to use torch.float32 when on an NPU device.

x_out = torch.view_as_real(x_out * freqs).flatten(2)
return x_out.to(x.dtype)

Expand Down
2 changes: 1 addition & 1 deletion diffsynth/utils/xfuser/xdit_context_parallel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def rope_apply(x, freqs, num_heads):
sp_rank = get_sequence_parallel_rank()
freqs = pad_freqs(freqs, s_per_rank * sp_size)
freqs_rank = freqs[(sp_rank * s_per_rank):((sp_rank + 1) * s_per_rank), :, :]
freqs_rank = freqs_rank.to(torch.complex64) if freqs_rank.device == "npu" else freqs_rank
freqs_rank = freqs_rank.to(torch.complex64) if freqs_rank.device.type == "npu" else freqs_rank
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the other file, while this change is correct, there's a potential mixed-precision issue on NPU devices.

x_out is created as complex128 on line 46. Here, freqs_rank is cast to complex64 for NPU devices.

During the multiplication x_out * freqs_rank on line 54, freqs_rank will likely be promoted back to complex128. This makes the cast to complex64 ineffective and might not resolve underlying performance or compatibility issues on NPU if complex128 is the problem.

To fully leverage complex64 on NPU, x_out should also be created as complex64 by changing x.to(torch.float64) on line 46 to use torch.float32 for NPU devices.

x_out = torch.view_as_real(x_out * freqs_rank).flatten(2)
return x_out.to(x.dtype)

Expand Down