diff options
| author | Galunid <karolek1231456@gmail.com> | 2024-05-30 13:40:00 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-30 21:40:00 +1000 |
| commit | 9c4c9cc83f7297a10bb3b2af54a22ac154fd5b20 (patch) | |
| tree | 81d201275f61a8e956d36d4cb5514c27332bbe05 /convert-hf-to-gguf.py | |
| parent | 59b0d077662fab430446b3119fa142f3291c45b2 (diff) | |
Move convert.py to examples/convert-legacy-llama.py (#7430)
* Move convert.py to examples/convert-no-torch.py
* Fix CI, scripts, readme files
* convert-no-torch -> convert-legacy-llama
* Move vocab thing to vocab.py
* Fix convert-no-torch -> convert-legacy-llama
* Fix lost convert.py in ci/run.sh
* Fix imports
* Fix gguf not imported correctly
* Fix flake8 complaints
* Fix check-requirements.sh
* Get rid of ADDED_TOKENS_FILE, FAST_TOKENIZER_FILE
* Review fixes
Diffstat (limited to 'convert-hf-to-gguf.py')
| -rwxr-xr-x | convert-hf-to-gguf.py | 4 |
1 files changed, 1 insertions, 3 deletions
diff --git a/convert-hf-to-gguf.py b/convert-hf-to-gguf.py index 98b50d15..9f29cda2 100755 --- a/convert-hf-to-gguf.py +++ b/convert-hf-to-gguf.py @@ -25,8 +25,6 @@ if 'NO_LOCAL_GGUF' not in os.environ: sys.path.insert(1, str(Path(__file__).parent / 'gguf-py')) import gguf -from convert import LlamaHfVocab - logger = logging.getLogger("hf-to-gguf") @@ -634,7 +632,7 @@ class Model: special_vocab.add_to_gguf(self.gguf_writer) def _set_vocab_llama_hf(self): - vocab = LlamaHfVocab(self.dir_model) + vocab = gguf.LlamaHfVocab(self.dir_model) tokens = [] scores = [] toktypes = [] |