python : add check-requirements.sh and GitHub workflow (#4585)
* python: add check-requirements.sh and GitHub workflow This script and workflow forces package versions to remain compatible across all convert*.py scripts, while allowing secondary convert scripts to import dependencies not wanted in convert.py. * Move requirements into ./requirements * Fail on "==" being used for package requirements (but can be suppressed) * Enforce "compatible release" syntax instead of == * Update workflow * Add upper version bound for transformers and protobuf * improve check-requirements.sh * small syntax change * don't remove venvs if nocleanup is passed * See if this fixes docker workflow * Move check-requirements.sh into ./scripts/ --------- Co-authored-by: Jared Van Bortel <jared@nomic.ai>
This commit is contained in:
parent
68eccbdc5b
commit
04ac0607e9
16 changed files with 360 additions and 130 deletions
2
requirements/requirements-convert-hf-to-gguf.txt
Normal file
2
requirements/requirements-convert-hf-to-gguf.txt
Normal file
|
@ -0,0 +1,2 @@
|
|||
-r ./requirements-convert.txt
|
||||
torch~=2.1.1
|
1
requirements/requirements-convert-llama-ggml-to-gguf.txt
Normal file
1
requirements/requirements-convert-llama-ggml-to-gguf.txt
Normal file
|
@ -0,0 +1 @@
|
|||
-r ./requirements-convert.txt
|
2
requirements/requirements-convert-lora-to-ggml.txt
Normal file
2
requirements/requirements-convert-lora-to-ggml.txt
Normal file
|
@ -0,0 +1,2 @@
|
|||
-r ./requirements-convert.txt
|
||||
torch~=2.1.1
|
2
requirements/requirements-convert-persimmon-to-gguf.txt
Normal file
2
requirements/requirements-convert-persimmon-to-gguf.txt
Normal file
|
@ -0,0 +1,2 @@
|
|||
-r ./requirements-convert.txt
|
||||
torch~=2.1.1
|
5
requirements/requirements-convert.txt
Normal file
5
requirements/requirements-convert.txt
Normal file
|
@ -0,0 +1,5 @@
|
|||
numpy~=1.24.4
|
||||
sentencepiece~=0.1.98
|
||||
transformers>=4.35.2,<5.0.0
|
||||
gguf>=0.1.0
|
||||
protobuf>=4.21.0,<5.0.0
|
Loading…
Add table
Add a link
Reference in a new issue