diff options
| author | Kawrakow <iwankawrakow@gmail.com> | 2024-10-01 12:28:29 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-10-01 12:28:29 +0300 |
| commit | 8457a26f83b2f6acd014449e91bfb60a37fcec0e (patch) | |
| tree | afcaefab2e5ae4f5d179a5b5543acd64f950caec /examples/pydantic_models_to_grammar.py | |
| parent | c2ff4f936a3060cb1ef6adc6e7c2664324c89d84 (diff) | |
CUDA: faster float -> iq4_nl conversion (#73)
* iqk_mul_mat: better iq4_nl implementation on Zen4/AVX2
PP-512 performance for LLaMA-3.1-8B goes to 162.6 t/s up
from 133.2 t/s.
* Speed up float -> iq4_nl conversion on CUDA
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Diffstat (limited to 'examples/pydantic_models_to_grammar.py')
0 files changed, 0 insertions, 0 deletions