diff options
| author | Kawrakow <iwankawrakow@gmail.com> | 2024-10-01 10:56:50 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-10-01 10:56:50 +0300 |
| commit | c2ff4f936a3060cb1ef6adc6e7c2664324c89d84 (patch) | |
| tree | 621e9012f130f4a6d852f464b49bca9151ef372b /examples/pydantic_models_to_grammar.py | |
| parent | 8cba4789da860d32cfc6d14f96ed37ade9e334bd (diff) | |
iqk_mul_mat: better iq4_nl implementation on Zen4/AVX2 (#72)
* iqk_mul_mat: better iq4_nl implementation on Zen4/AVX2
PP-512 performance for LLaMA-3.1-8B goes to 162.6 t/s up
from 133.2 t/s.
* Fix AVX2
In addition to fixing iq4_nl, it seems I never adhusted the AVX2
implementation for iq2_tn to the block scale removal?
This commit also fixes that.
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Diffstat (limited to 'examples/pydantic_models_to_grammar.py')
0 files changed, 0 insertions, 0 deletions