diff options
| author | jaime-m-p <167997752+jaime-m-p@users.noreply.github.com> | 2024-06-04 09:17:17 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-06-04 09:17:17 +0200 |
| commit | 3b38d48609280aa5f8ab7ea135a4351b2a5ee240 (patch) | |
| tree | c3fdcd91e9959ee394ea798d78731e611faaa008 /scripts | |
| parent | 6d1616944d9efd342ed2a4fd318722adfc9febcd (diff) | |
Per token attributes (#7685)
* Add per token attributes enum
* Using phi-3 for testing 'rstrip'
* Using jina-v2 for testing 'lstrip'
* Brute force test for 'lstrip' and 'rstrip'
* Implement 'rstrip' and 'lstrip'
* Update phi-3 GGUF file (obsolete since 917dc8c)
* Replace llama_token_type with llama_token_attribs
Diffstat (limited to 'scripts')
0 files changed, 0 insertions, 0 deletions