ver4a/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

llama : add thread safety test (#14035)

Browse source 
* llama : add thread safety test

* llamafile : remove global state

* llama : better LLAMA_SPLIT_MODE_NONE logic

when main_gpu < 0 GPU devices are not used

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
Diego Devesa 2025-06-16 08:11:43 -07:00 • committed by GitHub
parent 0dbcabde8c
commit 6adc3c3ebc
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 192 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

3
ggml/src/ggml-cpu/ggml-cpu-impl.h
View file

@ -503,6 +503,9 @@ static __m256 __lasx_xvreplfr2vr_s(const float val) {
// TODO: move to ggml-threading
void ggml_barrier(struct ggml_threadpool * tp);
void ggml_threadpool_chunk_set(struct ggml_threadpool * tp, int value);
int ggml_threadpool_chunk_add(struct ggml_threadpool * tp, int value);
#ifdef __cplusplus
}
#endif