diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2024-05-22 11:01:35 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-22 11:01:35 +0300 |
| commit | 3e5faa85032ec3106a2ad831bf412be9ff139f47 (patch) | |
| tree | dc85a6e015eecb4c771bffd6d3c4202459edaa9a /ggml-cuda | |
| parent | 201cc11afa0a1950e1f632390b2ac6c937a0d8f0 (diff) | |
cuda : fix rope + add tests (#7452)
* cuda : fix rope pos data
ggml-ci
* ggml : drop mode & 1 == 1 support for ggml_rope
ggml-ci
* ggml : support freq_factors for f16 rope (CPU)
ggml-ci
* tests : add rope tests using frequency factors
ggml-ci
Diffstat (limited to 'ggml-cuda')
| -rw-r--r-- | ggml-cuda/rope.cu | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/ggml-cuda/rope.cu b/ggml-cuda/rope.cu index 4a558f4b..50f2cf41 100644 --- a/ggml-cuda/rope.cu +++ b/ggml-cuda/rope.cu @@ -283,9 +283,9 @@ void ggml_cuda_op_rope(ggml_backend_cuda_context & ctx, ggml_tensor * dst) { const bool is_neox = mode & 2; const bool is_glm = mode & 4; - if (is_neox) { - pos = (const int32_t *) src1_d; + pos = (const int32_t *) src1_d; + if (is_neox) { if (src2 != nullptr) { freq_factors = (const float *) src2->data; } |