diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2024-02-17 23:04:16 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-02-17 23:04:16 +0200 |
| commit | 8f1be0d42f23016cb6819dbae01126699c4bd9bc (patch) | |
| tree | 4a142e745a73307190e9c5ef5c41aeb4aadaca7a /examples | |
| parent | 6e4e973b2615f8d390b1c4f4a7e05a119078bb0f (diff) | |
ggml : add ALiBi support for ggml_soft_max_ext (#5488)
* ggml : avoid recomputing alibi slopes (CPU)
* llama : reuse hparams.f_max_alibi_bias in all cases
ggml-ci
* ggml : support alibi bias in ggml_soft_max_ext (CPU + Metal)
ggml-ci
* ggml : handle all SRCs (do not break on first null)
ggml-ci
* tests : do not use slope for large soft_max
accumulates too much error
ggml-ci
* ggml : alternative ALiBi without extra tensor
We compute the slopes in the kernel
ggml-ci
* cuda : add ALiBi support in ggml_soft_max_ext
ggml-ci
* ggml : deprecate ggml_alibi
* ggml : support multi-sequence ALiBi (Metal)
ggml-ci
* cuda : add multi-seq ALiBi + remote F16 soft_max
ggml-ci
* ggml : update deprecation message
* ggml : fix pos ptr when no ALiBi
ggml-ci
* cuda : fix performance (pow -> powf)
* cuda : precompute ALiBi constants
* metal : pre-compute ALiBi slopes
ggml-ci
* llama : init kq_pos only if needed
ggml-ci
* test-backend-ops : add null pos test to soft_max
test-backend-ops : replace soft_max tests
ggml-ci
---------
Co-authored-by: slaren <slarengh@gmail.com>
Diffstat (limited to 'examples')
0 files changed, 0 insertions, 0 deletions