diff options
| author | 0cc4m <picard12@live.de> | 2024-03-05 13:33:42 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-03-05 13:33:42 +0100 |
| commit | 61d1c88e155515dd03940913a5707ea84a8b119b (patch) | |
| tree | c2c7de9900b33a73a6fba4299523b54528676e1f /ggml-vulkan.h | |
| parent | 21b08674331e1ea1b599f17c5ca91f0ed173be31 (diff) | |
Vulkan Improvements (#5835)
* Improve dequant shaders, add fast q4_0 dequant
* Optimize dmmv non-kquants for GCN
Remove unnecessary SPIR-V shader duplication
* Fix q4_0 dequant dispatch sizes
Fix backend free bug
* Optimize dequant shaders for q4_1, q5_0, q5_1 and q8_0
* Add unary and binary op shader templates
* Fix Vulkan check results
* Enable non-contiguous support for simple ops
* Add argsort
Basic q4_0 mmq shader and unit test
* Speed up q4_0 dequant code, enable mmq for q4_0
* Rework matmul pipeline selection
* Add soft_max alibi support
* Add q4_1, q5_0, q5_1 and q8_0 dequant mat mat mul shaders
* Add environment variable GGML_VK_FORCE_MAX_ALLOCATION_SIZE to limit max buffer size
Rename GGML_VULKAN_DISABLE_F16 to GGML_VK_DISABLE_F16 for consistency
Diffstat (limited to 'ggml-vulkan.h')
| -rw-r--r-- | ggml-vulkan.h | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/ggml-vulkan.h b/ggml-vulkan.h index 9645126b..e4317c3e 100644 --- a/ggml-vulkan.h +++ b/ggml-vulkan.h @@ -10,6 +10,7 @@ extern "C" { #define GGML_VK_NAME "Vulkan" #define GGML_VK_MAX_DEVICES 16 +GGML_API void ggml_vk_instance_init(void); GGML_API void ggml_vk_init_cpu_assist(void); GGML_API void ggml_vk_preallocate_buffers_graph_cpu_assist(struct ggml_tensor * node); |