diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2023-12-03 10:58:16 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-12-03 10:58:16 +0200 |
| commit | d7b800b8bc490a221acbd83c575206a907f2f6e2 (patch) | |
| tree | c41c5d8ead5fb3cb23ea0b5bca51f92a58e0d7cf /examples/server | |
| parent | 5a7d3125e7c24f223659b7f0b7aa7736986e92c0 (diff) | |
llama : pad KV cache size (#4280)
* llama : pad KV cache size to 32
* metal : try to improve batched decoding
Diffstat (limited to 'examples/server')
0 files changed, 0 insertions, 0 deletions