diff options
| author | Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com> | 2023-10-29 11:31:40 -0600 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-10-29 11:31:40 -0600 |
| commit | 6e08281e588bbba1a5d180290a94a43f167f3a1a (patch) | |
| tree | 46add394417eb2b5929793ca879c793a478fd3f8 /examples/server/server.cpp | |
| parent | 2046eb4345e62c4575b3cdc0115a51db89f3fb70 (diff) | |
Extend llama_kv_cache_seq_rm to allow matching any sequence (#3843)
* Extend llama_kv_cache_seq_rm to allow matichng any sequence
* Replace llama_kv_cache_tokens_rm with llama_kv_cache_clear
Use llama_kv_cache_clear for cache clearing
Change calls to llama_kv_cache_tokens_rm that want to delete by position to use llama_kv_cache_seq_rm functionality
Diffstat (limited to 'examples/server/server.cpp')
| -rw-r--r-- | examples/server/server.cpp | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/examples/server/server.cpp b/examples/server/server.cpp index 5b7e4139..c163c7f8 100644 --- a/examples/server/server.cpp +++ b/examples/server/server.cpp @@ -857,7 +857,7 @@ struct llama_server_context void kv_cache_clear() { // clear the entire KV cache - llama_kv_cache_tokens_rm(ctx, -1, -1); + llama_kv_cache_clear(ctx); clean_kv_cache = false; } |