b7815
ggml-cpu: aarm64: q5_K repack gemm and gemv (and generic) implementations (i8mm) (#18860)
- Boilerplate for q5_Kx8 REPACK on ARM and fallback
Signed-off-by: Alberto Cabrera alberto.cabrera@liquid.ai
- Implements make_block_q5_Kx8 by extending make_block_q4_Kx8
Signed-off-by: Alberto Cabrera alberto.cabrera@liquid.ai
q5_K repack gemm and gemv generics
Gemm and Gemv ARM implementations (i8mm)
Improved qh manipulation looking at non-repack vec_dot implementation
Full unroll
Apply Q5_K Gemv vand and vshl optimizations to gemm. Improve comments.
Signed-off-by: Alberto Cabrera alberto.cabrera@liquid.ai
- Fix wrong fallback definitions of Q5_K
Signed-off-by: Alberto Cabrera alberto.cabrera@liquid.ai
- Fixed comments. Reverted unnecessary formatting
Signed-off-by: Alberto Cabrera alberto.cabrera@liquid.ai
Fixed typo in generic definitions
Switching AND + Shift with Shift Insert. Better op interleaving.
Vectorize + unroll the block scales
Apply gemm optimizations to gemv
Improve bias calculation
Signed-off-by: Alberto Cabrera alberto.cabrera@liquid.ai
macOS/iOS:
Linux:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.1 DLLs
- Windows x64 (Vulkan)
- Windows x64 (SYCL)
- Windows x64 (HIP)
openEuler: