ver4a/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling (#12995)

Browse source 
* threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling

We talked about adding LOW priority for GGML threads in the original threadpool PR.
It might be useful for some cases to avoid contention.

Latest Windows ARM64 releases started parking (offlining) the CPU cores
more aggresively which results in suboptimal performance with n_threads > 4.
To deal with that we now disable Power Throttling for our threads for the NORMAL
and higher priorities.

Co-authored-by: Diego Devesa <slarengh@gmail.com>

* threading: disable SetThreadInfo() calls for older Windows versions

* Update tools/llama-bench/llama-bench.cpp

Co-authored-by: Diego Devesa <slarengh@gmail.com>

---------

Co-authored-by: Diego Devesa <slarengh@gmail.com>
This commit is contained in:
Max Krasnyansky 2025-05-31 15:39:19 -07:00 • committed by GitHub
parent b3a89c3d9e
commit 053b1539c0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 29 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tools/llama-bench/llama-bench.cpp
View file

@ -315,7 +315,7 @@ static void print_usage(int /* argc */, char ** argv) {
printf(" --numa <distribute|isolate|numactl> numa mode (default: disabled)\n");
printf(" -r, --repetitions <n> number of times to repeat each test (default: %d)\n",
cmd_params_defaults.reps);
printf(" --prio <0|1|2|3> process/thread priority (default: %d)\n",
printf(" --prio <-1|0|1|2|3> process/thread priority (default: %d)\n",
cmd_params_defaults.prio);
printf(" --delay <0...N> (seconds) delay between each test (default: %d)\n",
cmd_params_defaults.delay);