diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2024-01-17 15:45:03 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-01-17 15:45:03 +0200 |
| commit | 4f4bf35f46600441dec2f941e667291eeb9a18d8 (patch) | |
| tree | bcbb84ce3d43ab5788303312f8f5b35090292ad3 /examples/server/deps.sh | |
| parent | 2b3a665d3917edf393761a24c4835447894df74a (diff) | |
py : fix missing added_tokens_dict for SPM and BPE vocabs (#4971)
* py : fix missing added_tokens_dict for SPM vocab
* py : pad with unknown tokens when data is missing
ggml-ci
* py : fix BPE vocab conversion
ggml-ci
* py : fix padded dummy tokens (I hope)
Diffstat (limited to 'examples/server/deps.sh')
0 files changed, 0 insertions, 0 deletions