diff options
| author | 0cc4m <picard12@live.de> | 2024-03-29 17:29:21 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-03-29 17:29:21 +0100 |
| commit | ba0c7c70ab5b15f1f2be7fb0dfbe0366dda30d6c (patch) | |
| tree | 041a10dd587c26c42171be18e0f587f1fca2feca /examples/gguf-split/gguf-split.cpp | |
| parent | d48ccf3ad4fea5b9ede209c7f40be65371987bfe (diff) | |
Vulkan k-quant mmq and ggml-backend offload functionality (#6155)
* Fix Vulkan no kv offload incoherence
* Add k-quant mul mat mat shaders
* Rework working buffer allocation, reduces vram use noticeably
Clean up cpu assist code, replaced with ggml-backend offload function
* Default to all dedicated GPUs
* Add fallback for integrated GPUs if no dedicated GPUs are found
* Add debug info which device is allocating memory
* Fix Intel dequant issue
Fix validation issue
* Fix Vulkan GGML_OP_GET_ROWS implementation
* Clean up merge artifacts
* Remove Vulkan warning
Diffstat (limited to 'examples/gguf-split/gguf-split.cpp')
0 files changed, 0 insertions, 0 deletions