Intel Arc Vulkan prefill got a dedicated FA kernel in llama.cpp master
Reported by r/LocalLLaMA1 source ↓
A dedicated Vulkan Flash Attention prefill kernel for Intel Arc and Xe has been added to llama.cpp master. The community post reports faster prefill across tested GPUs and models; the kernel is enabled only for selected architectures.
Key points
- The kernel is enabled for ARL-H and Xe2/Xe3, but not the A770 because test results were inconsistent
- On Intel Arc B580, the post reports Gemma4 E4B prefill rising from 115.98 to 1,156.62 tokens per second
- Backend changes also cover single-KV-head models, including MQA and MLA
Details
The r/LocalLLaMA post reports that the kernel uses customized shaders for Intel matrix-core shapes and shared local memory. It is currently restricted to selected architectures; the post says the A770 is excluded because results were inconsistent.
In the post's B580 benchmark at a 16,384-token prompt, reported prefill throughput for Llama 3.1 8B rose from 347.79 to 633.73 tokens per second, while Gemma4 12B, a single-KV-head model, rose from 52 to 649. The post also gives results for ARL-H, PTL-H and B70 Pro systems, with gains varying by model and platform.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh