shalinib-ibm 3f3769ba76 ggml : Enable MMA for BF16 in llamafile_sgemm (#13148) ... This patch upstreams llamafile's cpu matrix multiplication kernels for ppc64le using MMA builtins for BF16 data type. This change results in 9x - 40x gains in total speed S t/s (ie all tokens/total time), across various batch sizes tested using llama-batched-bench benchmark. The patch is tested with Meta-Lllama-3-8B, and Mistral-7B models (BF16 models generated by using llama-quantize from corresponding FP32 models) on an IBM POWER10 machine. Signed-off-by: Shalini Salomi Bodapati <Shalini.Salomi.Bodapati@ibm.com>		2025-05-02 19:53:12 +03:00
..
sgemm.cpp	ggml : Enable MMA for BF16 in llamafile_sgemm (#13148)	2025-05-02 19:53:12 +03:00
sgemm.h	ggml : more perfo with llamafile tinyblas on x86_64 (#10714)	2024-12-24 18:54:49 +01:00