summaryrefslogtreecommitdiff
path: root/gguf-py/gguf/vocab.py
AgeCommit message (Expand)Author
2024-05-30Move convert.py to examples/convert-legacy-llama.py (#7430)Galunid
2024-05-08convert-hf : save memory with lazy evaluation (#7075)compilade
2024-05-03convert.py : add python logging instead of print() (#6511)Brian
2024-04-18convert : support models with multiple chat templates (#6588)Sigbjørn Skjæret
2024-02-15fix(gguf-py): special tokens are no longer skipped when add_<token>_token is ...Michaël de Vries
2023-12-21py : open merges file as 'utf-8' (#4566)howlger
2023-12-17gguf-py : fail fast on nonsensical special token IDs (#4489)Jared Van Bortel
2023-11-19gguf-py : export chat templates (#4125)slaren
2023-11-16Respect tokenizer.ggml.add_bos_token value when tokenizing (#4040)Kerfuffle
2023-11-11gguf-py: Refactor and allow reading/modifying existing GGUF files (#3981)Kerfuffle