What changed
17 builds from b9991 (continuing from b9990, covered July 13) through b10007 (July 14, 19:42 UTC). Developer-relevant highlights from this set:
| Build | UTC | Change |
|-------|-----|--------|
| b10007 | Jul 14, 19:42 | OpenCL: fix dp4a bug for devices where cl_khr_integer_dot_product is unavailable — prevents backend init failure |
| b10005 | Jul 14, 18:57 | DeepSeekV4: fix seq_rm — sequence removal and context update bug corrected |
| b10004 | Jul 14, 18:17 | Vulkan/CPU: f16 SET_ROWS support |
| b10003 | Jul 14, 17:42 | Tokenize tool migrated to common_params_parse |
| b10002 | Jul 14, 15:19 | New tensor contiguity checking functions |
| b10001 | Jul 14, 12:46 | Test utility: fix segfault on no-args invocation |
| b9999 | Jul 14, 11:23 | KleidiAI: add SME2 f32 kernel with dynamic scheduling (ARM) |
| b9996 | Jul 14, 10:41 | Flush log before exit to prevent message loss |
| b9995 | Jul 14, 09:44 | SYCL: set fattn_vec_nthreads=256 for Intel Arc Battlemage detection |
| b9994 | Jul 14, 05:29 | Metal: add Q2_0 support for Apple Silicon |
| b9993 | Jul 13, 23:09 | model: add Hy3 (hy_v3) / Tencent Hunyuan 3 with MTP speculative decoding |
TL;DR
b9993 adds Tencent Hunyuan 3 (open-weight MoE) support with multi-token prediction speculative decoding; b9994 adds Q2_0 quantization to Metal for Apple Silicon; b9999 adds ARM SME2 kernels; b10005 fixes a DeepSeekV4 sequence removal correctness bug.
Developer signal
Four items require action or attention: (1) b10005 — DeepSeekV4 seq_rm bug: If you serve DeepSeek V4-Pro or V4-Flash via llama.cpp and use sequence removal (e.g., KV cache eviction strategies in multi-user serving, context sliding, or the seq_rm API), this was a silent correctness bug affecting sequence copying and context updates. Upgrade to b10005+. (2) b9993 — Hunyuan 3 (hy_v3): Tencent's Hunyuan 3 open-weight MoE architecture is now supported, including MTP (multi-token prediction) speculative decoding. The architecture uses a MoE decoder with per-head Q/K RMSNorm, sigmoid router with expert selection bias, and a permanently-active ungated shared expert. GGUF weights are required; check the Hunyuan model card on Hugging Face for available quantizations. MTP speculative decoding can improve throughput 2–3× for supported models. (3) b9994 — Q2_0 Metal: Q2_0 quantization is now GPU-accelerated on Apple Silicon via Metal. If you are running aggressively quantized models on Apple hardware and hitting CPU offload for Q2_0 layers, this should push those operations to GPU. (4) b9999 — ARM SME2: For ARM CPUs with SME2 support (Cortex-X925 and newer), the new KleidiAI f32 kernel with dynamic scheduling is available. This targets on-device inference on the latest Android SoCs and Arm-based server chips. (5) b10007 — OpenCL dp4a: If you run on OpenCL and hit initialization failure due to cl_khr_integer_dot_product unavailability, upgrade to b10007.