ver4a/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

rpc : send hash when tensor data is above some fixed threshold (#12496)

Browse source 
* rpc : send hash when tensor data is above some fixed threshold

ref #10095

* rpc : put cache under $HOME/.cache/llama.cpp

* try to fix win32 build

* another try to fix win32 build

* remove llama as dependency
This commit is contained in:
Radoslav Gerganov 2025-03-28 08:18:04 +02:00 • committed by GitHub
parent 2099a9d5db
commit ab6ab8f809
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 290 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

4
ggml/include/ggml-rpc.h
View file

@ -17,7 +17,9 @@ GGML_BACKEND_API ggml_backend_buffer_type_t ggml_backend_rpc_buffer_type(const c
GGML_BACKEND_API void ggml_backend_rpc_get_device_memory(const char * endpoint, size_t * free, size_t * total);
GGML_BACKEND_API void ggml_backend_rpc_start_server(ggml_backend_t backend, const char * endpoint, size_t free_mem, size_t total_mem);
GGML_BACKEND_API void ggml_backend_rpc_start_server(ggml_backend_t backend, const char * endpoint,
const char * cache_dir,
size_t free_mem, size_t total_mem);
GGML_BACKEND_API ggml_backend_reg_t ggml_backend_rpc_reg(void);