diff options
| author | Leng Yue <lengyue@lengyue.me> | 2023-09-14 09:14:44 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-09-14 19:14:44 +0300 |
| commit | 35f73049af6c676a106a5a990a819ae0bc3fcd7d (patch) | |
| tree | 55807c47e621aca6ffe3cb8936ade0f3f80e2921 /llama.cpp | |
| parent | 71ca2fad7d6c0ef95ef9944fb3a1a843e481f314 (diff) | |
speculative : add heuristic algorithm (#3006)
* Add heuristic algo for speculative
* Constrain minimum n_draft to 2
* speculative : improve heuristic impl
* speculative : be more rewarding upon guessing max drafted tokens
* speculative : fix typos
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Diffstat (limited to 'llama.cpp')
0 files changed, 0 insertions, 0 deletions