diff options
| author | Kawrakow <iwankawrakow@gmail.com> | 2024-10-02 17:05:56 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-10-02 17:05:56 +0300 |
| commit | 50b5e90112766dc4de276ccb0d0abf0f9a974b84 (patch) | |
| tree | b463969276dad675ece86f4ecca1ebb001ba5019 /ggml/include | |
| parent | cce49832c1b81b4e535e78ff308417ef3a386b18 (diff) | |
Fused unary(x)*y (#70)
* Adding fused y*unary(x) op
* Fused y*unary(x) op: CUDA
* Fused y*unary(x) op: dedicated CPU implementation for silu and gelu
* Fused y*unary(x) op: Metal
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Diffstat (limited to 'ggml/include')
| -rw-r--r-- | ggml/include/ggml.h | 13 |
1 files changed, 13 insertions, 0 deletions
diff --git a/ggml/include/ggml.h b/ggml/include/ggml.h index 08fe6a3e..b1aebd21 100644 --- a/ggml/include/ggml.h +++ b/ggml/include/ggml.h @@ -487,6 +487,7 @@ extern "C" { GGML_OP_RMS_NORM_BACK, GGML_OP_GROUP_NORM, GGML_OP_FUSED_RMS_NORM, + GGML_OP_FUSED_MUL_UNARY, GGML_OP_MUL_MAT, GGML_OP_MUL_MAT_ID, @@ -963,6 +964,18 @@ extern "C" { struct ggml_tensor * a, struct ggml_tensor * b); + GGML_API struct ggml_tensor * ggml_fused_mul_unary( + struct ggml_context * ctx, + struct ggml_tensor * a, + struct ggml_tensor * b, + enum ggml_unary_op op); + + GGML_API struct ggml_tensor * ggml_fused_mul_unary_inplace( + struct ggml_context * ctx, + struct ggml_tensor * a, + struct ggml_tensor * b, + enum ggml_unary_op op); + GGML_API struct ggml_tensor * ggml_div( struct ggml_context * ctx, struct ggml_tensor * a, |