diff options
| author | Michael de Gans <michael.john.degans@gmail.com> | 2024-06-19 22:32:01 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-06-20 08:32:01 +0300 |
| commit | 2075a66a96cc1b04eabec7cf4b3051193d6f719e (patch) | |
| tree | 4b4e30b099940c6aae4321a6394c2f15ab237bc3 | |
| parent | ba5899315283eb1df3902363daf79bdc5eefe426 (diff) | |
metal : fix `ggml_metal_supports_op` for BF16 (#8021)
Currently the Metal backend does not support BF16. `ggml_metal_supports_op` was returning true in these cases, leading to a crash with models converted with `--leave-output-tensor`. This commit checks if the first few sources types are BF16 and returns false if that's the case.
| -rw-r--r-- | ggml-metal.m | 6 |
1 files changed, 6 insertions, 0 deletions
diff --git a/ggml-metal.m b/ggml-metal.m index f894274c..79902c9a 100644 --- a/ggml-metal.m +++ b/ggml-metal.m @@ -735,6 +735,12 @@ static id<MTLBuffer> ggml_metal_get_buffer(struct ggml_tensor * t, size_t * offs } static bool ggml_metal_supports_op(const struct ggml_metal_context * ctx, const struct ggml_tensor * op) { + for (size_t i = 0, n = 3; i < n; ++i) { + if (op->src[i] != NULL && op->src[i]->type == GGML_TYPE_BF16) { + return false; + } + } + switch (op->op) { case GGML_OP_UNARY: switch (ggml_get_unary_op(op)) { |