diff options
| author | Iwan Kawrakow <iwan.kawrakow@gmail.com> | 2024-06-18 12:00:16 +0300 |
|---|---|---|
| committer | Iwan Kawrakow <iwan.kawrakow@gmail.com> | 2024-06-22 12:02:52 +0300 |
| commit | 8c6276f6a1c6d9d82b5f0114d838fcc4f277234a (patch) | |
| tree | 6f80e1f3b1017d3d3e88082db0399c6884bc9725 /ggml-cuda/template-instances | |
| parent | 1de6476d751a02978b035feb38066462c4382877 (diff) | |
Bitnet: 2.25 bpw version
Just scaler and AVX2 for now.
PP-512 is even faster (325 t/s on the Ryzn-7950X, 404 t/s on
Ryzen-5975WX). We lose ~6-7% for TG due to being memory bound and
the model being 10% larger.
Diffstat (limited to 'ggml-cuda/template-instances')
0 files changed, 0 insertions, 0 deletions