ver4a/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update Makefile var + add comment

Browse source
This commit is contained in:
Georgi Gerganov 2023-03-11 12:26:16 +02:00
parent a5c5ae2f54
commit 7211862c94
No known key found for this signature in database
GPG key ID: 449E073F9DC10735
2 changed files with 10 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

8
README.md
View file

@ -180,5 +180,11 @@ When running the larger models, make sure you have enough disk space to store al
In general, it seems to work, but I think it fails for unicode character support. Hopefully, someone can help with that
- I don't know yet how much the quantization affects the quality of the generated text
- Probably the token sampling can be improved
- x86 quantization support [not yet ready](https://github.com/ggerganov/ggml/pull/27). Basically, you want to run this on Apple Silicon. For now, on Linux and Windows you can use the F16 `ggml-model-f16.bin` model, but it will be much slower.
- x86 quantization support [not yet ready](https://github.com/ggerganov/ggml/pull/27). Basically, you want to run this
on Apple Silicon. For now, on Linux and Windows you can use the F16 `ggml-model-f16.bin` model, but it will be much
slower.
- The Accelerate framework is actually currently unused since I found that for tensors shapes typical for the Decoder,
there is no benefit compared to the ARM_NEON intrinsics implementation. Of course, it's possible that I simlpy don't
know how to utilize it properly. But in any case, you can even disable it with `LLAMA_NO_ACCELERATE=1 make` and the
performance will be the same, since no BLAS calls are invoked by the current implementation