diff options
| author | Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com> | 2023-10-06 10:10:13 -0600 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-10-06 10:10:13 -0600 |
| commit | 9ca79d5cbbc8d43f2bff951404b6a40ff1ee3788 (patch) | |
| tree | d53db5e2b2c7d47d38c4ae4b500d02bf8d624c6c /examples/server | |
| parent | 0c731ca4039ccff86ffab90eaae4ca98037c4496 (diff) | |
kv cache slot search improvements (#3493)
* kv cache slot search improvements
* Use n_ctx in kv find slot for consistency
* Ensure kv cache head points to a valid slot in llama_decode internal
* Add some comments to prevent dumb people (like me) from getting confused.
Diffstat (limited to 'examples/server')
0 files changed, 0 insertions, 0 deletions