vulkan: disable MMVQ on AMD UMA devices

[winstonma]

MMVQ's quantization overhead outweighs bandwidth savings on UMA where CPU and GPU share the same memory pool.

Overview

This is the testing result before the patch is written (tested on AMD UMA only):

# Force large workgroup for DMMV
❯ GGML_VK_FORCE_MMVQ=1 llama-bench --model ~/model/gemma-4-26B-A4B-it-qat-GGUF/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -p 0 -n 128 -r 5
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 880M Graphics (RADV STRIX1) (radv) | uma: 1 | fp16: dot2 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |           tg128 |         28.78 ± 0.10 |

# Disable MMVQ entirely (uses plain fp16 dequant path)  
❯ GGML_VK_DISABLE_MMVQ=1 llama-bench --model ~/model/gemma-4-26B-A4B-it-qat-GGUF/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -p 0 -n 128 -r 5
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 880M Graphics (RADV STRIX1) (radv) | uma: 1 | fp16: dot2 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| gemma4 26B.A4B Q4_0            |  13.26 GiB |    25.23 B | Vulkan     |  -1 |           tg128 |         30.79 ± 0.04 |

tg value jump from 28.78 to 30.79 on my device.

The patch is applied to AMD path because this is the device that I could test.

Additional information

The following explanation is written by Claude:

The MMVQ path quantizes the input vector to Q8_1 before dispatching, which adds CPU-side work and an extra quantization kernel on the GPU. On a discrete GPU with fast VRAM, that overhead is worth it because it reduces the weight-loading bandwidth. On my AMD UMA device, the CPU and GPU share the same LPDDR5X pool — the quantization step costs real time and the bandwidth savings are smaller because my device is already memory-bandwidth-limited from both sides simultaneously.

The effectiveness of the patch depends on how much time each model actually spends in these kernels relative to everything else:

• Gemma 4 sparse MoE with very few active experts per token (top-2 out of 128 in its case). Each expert matmul has a very small weight matrix — the k dimension of each expert is tiny. That means the quantization overhead of MMVQ is large relative to the actual compute work, so removing it is felt immediately.
• Dense models have large weight matrices where bandwidth dominates regardless, so whether MMVQ is on or off makes little difference because the bottleneck isn't the quantization step but the weight read itself.

Based on the test I see that this patch would

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes, suggesting and patch writing

[0cc4m]

Your benchmark is invalid, you don't even show the baseline. FORCE_MMVQ enables it anywhere, which is not default behaviour. Also, here's a counter example on AMD Strix Halo:

before is just base master, after is with GGML_VK_DISABLE_MMVQ=1

model	size	params	ngl	fa	mmap	test	t/s (before)	t/s (after)	diff
llama 8B Q4_0	4.33 GiB	8.03 B	-1	1	0	tg128	44.52 ± 0.08	43.32 ± 0.06	-2.7%
llama 8B Q4_K - Small	4.36 GiB	8.03 B	-1	1	0	tg128	44.21 ± 0.12	42.60 ± 0.10	-3.6%
qwen35moe 35B.A3B Q6_K	27.98 GiB	34.66 B	-1	1	0	tg128	53.77 ± 0.28	53.54 ± 0.35	-0.4%
qwen35 27B Q6_K	21.62 GiB	26.90 B	-1	1	0	tg128	8.93 ± 0.00	8.93 ± 0.00	+0.0%