Georgi Gerganov
|
f3f65429c4
|
llama : reorganize source code + improve CMake (#8006)
* scripts : update sync [no ci]
* files : relocate [no ci]
* ci : disable kompute build [no ci]
* cmake : fixes [no ci]
* server : fix mingw build
ggml-ci
* cmake : minor [no ci]
* cmake : link math library [no ci]
* cmake : build normal ggml library (not object library) [no ci]
* cmake : fix kompute build
ggml-ci
* make,cmake : fix LLAMA_CUDA + replace GGML_CDEF_PRIVATE
ggml-ci
* move public backend headers to the public include directory (#8122)
* move public backend headers to the public include directory
* nix test
* spm : fix metal header
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* scripts : fix sync paths [no ci]
* scripts : sync ggml-blas.h [no ci]
---------
Co-authored-by: slaren <slarengh@gmail.com>
|
2024-06-26 18:33:02 +03:00 |
|
Johannes Gäßler
|
750f60c03e
|
CUDA: fix Pascal FA, deq. KV to FP16 for batch > 8 (#7681)
|
2024-06-01 15:47:04 +02:00 |
|
Johannes Gäßler
|
9b596417af
|
CUDA: quantized KV support for FA vec (#7527)
* CUDA: quantized KV support for FA vec
* try CI fix
* fix commented-out kernel variants
* add q8_0 q4_0 tests
* fix nwarps > batch size
* split fattn compile via extern templates
* fix flake8
* fix metal tests
* fix cmake
* make generate_cu_files.py executable
* add autogenerated .cu files
* fix AMD
* error if type_v != FP16 and not flash_attn
* remove obsolete code
|
2024-06-01 08:44:14 +02:00 |
|
Johannes Gäßler
|
cd93a28cb1
|
CUDA: fix FA out-of-bounds reads (#7479)
|
2024-05-23 00:31:20 +02:00 |
|
Johannes Gäßler
|
38c03478a3
|
CUDA: fix FA out-of-bounds writes (#7465)
|
2024-05-22 17:58:25 +02:00 |
|
Georgi Gerganov
|
9b3d833189
|
cuda : fix compile warning (#7454)
|
2024-05-22 12:36:37 +03:00 |
|
Johannes Gäßler
|
95fb0aefab
|
CUDA: remove incorrect precision check (#7454)
|
2024-05-22 10:24:29 +02:00 |
|
Johannes Gäßler
|
133d99c599
|
CUDA: deduplicate FlashAttention code (#7352)
|
2024-05-18 12:36:25 +02:00 |
|
Johannes Gäßler
|
0fc1e820a9
|
CUDA: faster large batch FA without tensor cores (#7314)
|
2024-05-17 18:54:52 +02:00 |
|