diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2024-01-08 11:14:04 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-01-08 11:14:04 +0200 |
| commit | b0034d93ce2949ce7d9c098ca02e56f66cd484e2 (patch) | |
| tree | 5409bc6706ff5cf5aabc109ea466b73d46bb9839 /examples/batched | |
| parent | b7e7982953f80a656e03feb5cfb17a17a173eb26 (diff) | |
examples : add passkey test (#3856)
* examples : add passkey test
* passkey : better prints
* passkey : select pass key pos from CLI
* passkey : simplify n_past logic
* make : add passkey target
* passkey : add "self-extend"-like context extension (#4810)
* llama : "self-extend"-like context extension
* passkey : add comment
* passkey : add readme
Diffstat (limited to 'examples/batched')
| -rw-r--r-- | examples/batched/batched.cpp | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/examples/batched/batched.cpp b/examples/batched/batched.cpp index 22a4265d..b1775e0b 100644 --- a/examples/batched/batched.cpp +++ b/examples/batched/batched.cpp @@ -69,6 +69,7 @@ int main(int argc, char ** argv) { std::vector<llama_token> tokens_list; tokens_list = ::llama_tokenize(model, params.prompt, true); + const int n_kv_req = tokens_list.size() + (n_len - tokens_list.size())*n_parallel; // initialize the context |