How to Use Z-Image on a GPU with Only 4GB VRAM

[leejet]

Wiki: https://github.com/leejet/stable-diffusion.cpp/wiki/How-to-Use-Z%E2%80%90Image-on-a-GPU-with-Only-4GB-VRAM

[stduhpf]

Just a small remark: The text encoder shipped with Z-Image is not off-the-shelf Qwen3-4B-Instruct-2507. It is not base Qwen3-4B either. I believe (just a hunch) it's its own thing, they probably continued pretraining it from some variant of Qwen3 4B alongside the image gen model.

Below is a comparison between Qwen3-4B-Instruct-2507 and the "qwen_3_4B" model shipped with z-image, with f16 to avoid imatrix causing varience and in GGUF format for a fair apple-to-apple comparison.

Qwen3-4B-Instruct-2507-F16.gguf	Qwen_3_4b-FP16.gguf

[stduhpf]

Nevermind, I now think it is just base Qwen3-4B (not to be confused with Qwen3-4B-Base). Not sure why in my earlier testing I was under the imporession it was a different thing.

Qwen3-4B-f16.gguf	Qwen_3_4b-FP16.gguf

[stduhpf]

I can run it at 20s/it for 512x512 resolution on my old RX 580 4GB using Vulkan on Linux. Not bad at all.

[netrunnereve]

That's suspiciously slow, on my RX 470 with Vulkan on Linux I get 7s/it at 512x512 and 35 s/it at 1024x1024 with the Q4_K model which just fits in 4GB with clip on CPU. On my 8GB card I use the Q8_0 model.

I'm using diffusion fa and vae conv direct but that shouldn't make that much of a difference.

[stduhpf]

How? I'm using Q3_K diffusion model, but that shouldn't be that much slower than Q4_k... I'm also running it with diffusion fa and vae conv direct. I was happy with my 20s/it until I read your comment.

Could be a driver issue?

Edit: with --type q4_K I get 17s/it, which is significantly faster, but still much slower than your 7s/it...
Edit 2: Reinstalled mesa drivers and Vulkan sdk, now I get 16s/it with the Q3_K model

[netrunnereve]

I just tested with Q3_K and I'm getting 7.5s/it. Something's not right and I know from when I did the Vulkan kquant optimizations that Q3 isn't that much slower than the Q4 stuff.

Are you using the default samplers? Also try to run with RADV_PERFTEST=nogttspill if your Mesa is new enough and check your clocks and vram/gtt usage with radeontop.

[stduhpf]

Also try to run with RADV_PERFTEST=nogttspill if your Mesa is new enough and check your clocks and vram/gtt usage with radeontop.

That worked! I get only ~7.3 s/it when I set this env variable.

[leejet]

On my RTX 3050 Laptop GPU, for generating 512x512 image, I can achieve 2.17s/it.

[Green-Sky]

Not very low spec, but still aging: RTX 2070, 512x512 with Q3_K_M, I get 1.19s/it.

[Green-Sky]

Worth of note is the speed of qwen on my cpu:

[DEBUG] ggml_extend.hpp:1689 - qwen3 compute buffer size: 1.86 MB(RAM)
[DEBUG] conditioner.hpp:1896 - computing condition graph completed, taking 823 ms
[INFO ] stable-diffusion.cpp:2917 - get_learned_condition completed, taking 824 ms