diff options
| author | Kawrakow <iwankawrakow@gmail.com> | 2025-03-21 07:23:36 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-03-21 07:23:36 +0100 |
| commit | b8d1fac97b756968b86b470d44bb1026ded7157a (patch) | |
| tree | 5a5893796293475185e833a787648830a7189450 /ggml/src/ggml-kompute.cpp | |
| parent | 127c6ee6493a3084995d754d987f0240ffdffe6a (diff) | |
Convert models to row-interleaved quants using the quantize tool (#272)
* Repack a model with the quantize tool
* WIP
* Fixed various issues
As we don't have a way to tell if a repacked quant has been modified,
I had to remove the modification at the expense of a slight decrease
in performance. This affects q8_0_r8, q8_KV_r8, q8_k_r8 on Zen4, and
q4_0_r8 on ARM.
* Create wk_b and wv_b as Q8_0_R8 if the wkv_b type is interleaved
* Fix GCC 13.3 compilation error
* Another one
* Add missing include
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Diffstat (limited to 'ggml/src/ggml-kompute.cpp')
0 files changed, 0 insertions, 0 deletions