diff options
| author | saood06 <saood05@gmail.com> | 2025-06-06 03:33:47 -0500 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-06-06 11:33:47 +0300 |
| commit | ffd87f282e76ff9d34f47efd6d3f6af2071d416a (patch) | |
| tree | f593753dd699b03ba50fc42a71214ca614f1f517 /examples/gbnf-validator | |
| parent | eded4e20d4decdc6e8c18e645fd1db0833ad251d (diff) | |
Make prompt cache saving and restoring MLA aware (#497)
* Remove kv_l, kvt_l and just use k_l and v_l
* Hopefully take care of missing V cache (MLA)
* Fix save and restore when there is no V cache
* Fix double print
* Update write_kv_cache_data and read_kv_cache_data to be MLA aware
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Diffstat (limited to 'examples/gbnf-validator')
0 files changed, 0 insertions, 0 deletions