diff options
| author | Kawrakow <iwankawrakow@gmail.com> | 2025-02-11 14:46:30 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-02-11 14:46:30 +0200 |
| commit | 3c98bfb33d149a0d9d3bb91604dd12709721e3cf (patch) | |
| tree | 6a1e5fc373032bb18a62ec3616625eedf1a9f1f3 /examples/imatrix/imatrix.cpp | |
| parent | a366a3d17d8f2de0eb8c3d9eddc7b5840fb5761a (diff) | |
DeepSeek FA support (CPU only) (#200)
* Adding support for K head size != V head size
This is relevant for DeepSeek models.
At this point ggml CPU FA works.
Now I need to go and change iqk FA to make it work
with Dk != Dv.
* iqk support for K head size != V head size
To not have compilation time explode, just
Dk = 192, Dv = 128 for now (DeepSeek)
* FA: very slightly faster for nq = 1 (TG)
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Diffstat (limited to 'examples/imatrix/imatrix.cpp')
0 files changed, 0 insertions, 0 deletions