diff options
| author | Iwan Kawrakow <iwan.kawrakow@gmail.com> | 2024-06-19 16:46:51 +0200 |
|---|---|---|
| committer | Iwan Kawrakow <iwan.kawrakow@gmail.com> | 2024-06-22 12:02:52 +0300 |
| commit | 257fa740148293d69aaaeeca5b22450221e34ea4 (patch) | |
| tree | 577f5e609086ce5018b773256f5605487ded3d51 /ggml.c | |
| parent | a2e43b83c9344e7c1130e3e95917bdd61dfb6aab (diff) | |
bitnet(scale in a separate tensor): Metal
iq2_bn TG-128 drops to 84 t/s, while I see in the logs
that we had 97 t/s. If true, that's a pretty massive
performance penalty for TG. Let me guess: ggml_mul is not
exactly the most performant operation on Metal.
Diffstat (limited to 'ggml.c')
0 files changed, 0 insertions, 0 deletions