diff options
| author | Kawrakow <iwankawrakow@gmail.com> | 2025-06-11 11:12:30 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-06-11 11:12:30 +0300 |
| commit | e56061fa12e5fcb7a9a8fa5fca9e74c20d166bcc (patch) | |
| tree | f0627d22fac777db7e397119b2f38421a2b289b8 /docs | |
| parent | 3c1f2c68fdbb9e5be26aad85902c3d86057d69b5 (diff) | |
IQ2_XXS: much faster CPU prompt processing (#515)
* Much faster iq2_xxs GEMM
PP-512 = 290 t/s vs ~110 t/s (iq2_xxs) or 148 t/s (iq2_xxs_r4) on main.
* iq2_xxs: q8_2_x4 GEMM
* iq2_xxs: use template for q8_2_x4 GEMM
* Fix AVX2
* Cleanup
* NEON is not working yet, so still use Q8_K GEMM
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions