diff options
| author | Kawrakow <48489457+ikawrakow@users.noreply.github.com> | 2023-08-29 23:55:03 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-08-29 23:55:03 +0300 |
| commit | e37e69dcc3d52f21222a63cafed2a71b3f6b53c6 (patch) | |
| tree | 887b453350fd897b3d1114a3924a2f50ecbb8da8 /examples/perplexity/perplexity.cpp | |
| parent | 53885d7256909ec3e2176cdc2477f3986c15ec69 (diff) | |
10X faster BPE tokenizer (#2876)
* 10X faster BPE tokenizer
* Remove comment that no longer applies
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Diffstat (limited to 'examples/perplexity/perplexity.cpp')
0 files changed, 0 insertions, 0 deletions