diff options
| author | 0cc4m <picard12@live.de> | 2024-03-05 13:33:42 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-03-05 13:33:42 +0100 |
| commit | 61d1c88e155515dd03940913a5707ea84a8b119b (patch) | |
| tree | c2c7de9900b33a73a6fba4299523b54528676e1f /grammars/json_arr.gbnf | |
| parent | 21b08674331e1ea1b599f17c5ca91f0ed173be31 (diff) | |
Vulkan Improvements (#5835)
* Improve dequant shaders, add fast q4_0 dequant
* Optimize dmmv non-kquants for GCN
Remove unnecessary SPIR-V shader duplication
* Fix q4_0 dequant dispatch sizes
Fix backend free bug
* Optimize dequant shaders for q4_1, q5_0, q5_1 and q8_0
* Add unary and binary op shader templates
* Fix Vulkan check results
* Enable non-contiguous support for simple ops
* Add argsort
Basic q4_0 mmq shader and unit test
* Speed up q4_0 dequant code, enable mmq for q4_0
* Rework matmul pipeline selection
* Add soft_max alibi support
* Add q4_1, q5_0, q5_1 and q8_0 dequant mat mat mul shaders
* Add environment variable GGML_VK_FORCE_MAX_ALLOCATION_SIZE to limit max buffer size
Rename GGML_VULKAN_DISABLE_F16 to GGML_VK_DISABLE_F16 for consistency
Diffstat (limited to 'grammars/json_arr.gbnf')
0 files changed, 0 insertions, 0 deletions