What changed
5 builds on July 9 following the July 8 security sprint (which ended at b9929). Focus shifts to per-backend math optimization: AMD HIP gains -ffast-math flag parity with CUDA; Apple Metal adds depthwise 2D convolution; CUDA fusion matcher is realigned.
| Build | UTC | Key change |
|-------|-----|------------|
| b9940 | 11:38 | llama-bench: fix offline mode initialization crash |
| b9939 | 10:58 | Metal: add CONV_2D_DW (depthwise 2D convolution) F16 support |
| b9938 | 10:08 | HIP: enable -ffast-math for AMD GPU speed parity with CUDA |
| b9937 | 08:30 | CUDA: realign fusion matcher type predicates and contiguity validation |
| b9936 | 07:09 | server: enforce minimum step constraints in batch processing |
TL;DR
5 builds in ~5h; b9938 brings -ffast-math to AMD HIP (matches CUDA's optimization flag, typically 5โ15% throughput improvement on matrix ops); b9939 adds Metal CONV_2D_DW depthwise convolution on Apple Silicon.
Developer signal
Upgrade to b9940 (cumulative). Two actionable notes: (1) AMD/HIP -ffast-math (b9938): AMD GPU users on ROCm builds should see throughput gains consistent with CUDA's -ffast-math benefit โ historically 5โ15% on matrix multiply workloads. -ffast-math trades IEEE floating-point strictness for speed; if you have downstream tasks that depend on exact floating-point reproducibility, benchmark before committing to this binary. For most LLM inference workloads, the accuracy impact is negligible. (2) Metal CONV_2D_DW (b9939): adds depthwise 2D convolution kernel support on Apple Silicon via Metal, enabling models that use depthwise convolution layers (common in lightweight vision encoder architectures) to fully offload to the Metal GPU. Previously those layers fell back to CPU on Mac.