diff options
| author | Tameem <113388789+AhmadTameem@users.noreply.github.com> | 2023-10-03 23:38:19 +0500 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-10-03 21:38:19 +0300 |
| commit | 79f34abddb72ac5ddbf118f3d87520b611a10a7d (patch) | |
| tree | a3c46ea79eebaa697d902370cf2c295910166f39 /tests/test-quantize-perf.cpp | |
| parent | 8186242b6d67cf87ae179fb1a62f52fdf0e5c5eb (diff) | |
ggml : add RISC-V Vector Support for K-Quants and improved the existing intrinsics (#3453)
* Added RVV intrinsics support for Q8 quantize row and also improved the existing dot product function for risc-v.
The RVV intrinsics is added for the following quantize row functions
quantize_row_q8_0
quantize_row_q8_1
The following dot product functions have also been optimized by using LMUL = 1/2 instead of LMUL = 1
ggml_vec_dot_q4_0_q8_0
ggml_vec_dot_q4_1_q8_1
ggml_vec_dot_q5_0_q8_0
ggml_vec_dot_q5_1_q8_1
And vector initialization in Q5 by temporary array is also replaced by the vid intrinsics
Signed-off-by: Ahmad Tameem <ahmad.tameem@10xengineers.ai>
* Added RVV intrinsics support for k_quants
This adds RISC-V Vector intrinsics support for the following K_quants functions for both QKK = 256 and QKK = 64
ggml_vec_dot_q2_K_q8_K
ggml_vec_dot_q3_K_q8_K
ggml_vec_dot_q4_K_q8_K
ggml_vec_dot_q5_K_q8_K
ggml_vec_dot_q6_K_q8_K
Signed-off-by: Ahmad Tameem <ahmad.tameem@10xengineers.ai>
---------
Signed-off-by: Ahmad Tameem <ahmad.tameem@10xengineers.ai>
Diffstat (limited to 'tests/test-quantize-perf.cpp')
0 files changed, 0 insertions, 0 deletions