diff options
| author | Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com> | 2023-10-29 11:31:40 -0600 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-10-29 11:31:40 -0600 |
| commit | 6e08281e588bbba1a5d180290a94a43f167f3a1a (patch) | |
| tree | 46add394417eb2b5929793ca879c793a478fd3f8 /examples/llama-bench/llama-bench.cpp | |
| parent | 2046eb4345e62c4575b3cdc0115a51db89f3fb70 (diff) | |
Extend llama_kv_cache_seq_rm to allow matching any sequence (#3843)
* Extend llama_kv_cache_seq_rm to allow matichng any sequence
* Replace llama_kv_cache_tokens_rm with llama_kv_cache_clear
Use llama_kv_cache_clear for cache clearing
Change calls to llama_kv_cache_tokens_rm that want to delete by position to use llama_kv_cache_seq_rm functionality
Diffstat (limited to 'examples/llama-bench/llama-bench.cpp')
| -rw-r--r-- | examples/llama-bench/llama-bench.cpp | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/examples/llama-bench/llama-bench.cpp b/examples/llama-bench/llama-bench.cpp index 20767d55..78039818 100644 --- a/examples/llama-bench/llama-bench.cpp +++ b/examples/llama-bench/llama-bench.cpp @@ -1037,7 +1037,7 @@ int main(int argc, char ** argv) { test t(inst, lmodel, ctx); - llama_kv_cache_tokens_rm(ctx, -1, -1); + llama_kv_cache_clear(ctx); // warmup run if (t.n_prompt > 0) { @@ -1048,7 +1048,7 @@ int main(int argc, char ** argv) { } for (int i = 0; i < params.reps; i++) { - llama_kv_cache_tokens_rm(ctx, -1, -1); + llama_kv_cache_clear(ctx); uint64_t t_start = get_time_ns(); if (t.n_prompt > 0) { |