diff options
author | xaedes <xaedes@gmail.com> | 2023-11-07 09:04:51 +0100 |
---|---|---|
committer | GitHub <noreply@github.com> | 2023-11-07 10:04:51 +0200 |
commit | e9c1cecb9d7d743d30b4a29ecd56a411437def0a (patch) | |
tree | 8d7382389695181bf2f91fbbfd6fbb424d625f8b /examples | |
parent | 54b4df8886103b436a4bb3b60f4d84824f9e8868 (diff) |
ggml : fix backward rope after YaRN (#3974)
* fix backward process of rope
rope backward process was broken after YaRN RoPE (#2268) implementation, due to missing changes in backward functions.
the code for the backward process is nearly identically to the forward process:
the only difference is the sign of the sin-values.
to avoid future regressions remove the near-duplicate backward functions and reuse the forward code:
for this a new function argument `bool forward` was added to `ggml_compute_forward_rope_f32` and `ggml_compute_forward_rope_f16`.
the sin-values will be negated when forward is false.
* fix finetune rope call to use correct default attn_factor of 1.0f
* remove unused `ggml_rope_xpos_back`
it is better to have only one `ggml_rope_back` function that accepts all rope parameters, so that `ggml_compute_backward` can propagate all parameters without having to switch between different rope_back variants.
* fix comments explaining the sinus sign in ggml_forward_rope
* add missing function arguments in declaration
* fix function argument type in declaration
Diffstat (limited to 'examples')
-rw-r--r-- | examples/finetune/finetune.cpp | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/examples/finetune/finetune.cpp b/examples/finetune/finetune.cpp index 649a3b7c..fa7dbe49 100644 --- a/examples/finetune/finetune.cpp +++ b/examples/finetune/finetune.cpp @@ -643,7 +643,7 @@ static struct ggml_tensor * llama_build_lora_finetune_graphs( return ggml_rope_custom(ctx, t, KQ_pos, n_rot, rope_mode, n_ctx, 0, - rope_freq_base, rope_freq_scale, 0.0f, 0.0f, 0.0f, 0.0f + rope_freq_base, rope_freq_scale, 0.0f, 1.0f, 0.0f, 0.0f ); }; |