diff options
| author | slaren <slarengh@gmail.com> | 2023-12-24 14:34:22 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-12-24 14:34:22 +0100 |
| commit | 5bf3953d7e9831ea22b0bc017ce97409b801ccf1 (patch) | |
| tree | 48c0136d9943fb9cca22209894464970549c24b5 /gguf-py/gguf/tensor_mapping.py | |
| parent | 708e179e8562c2604240df95a2241dea17fd808b (diff) | |
cuda : improve cuda pool efficiency using virtual memory (#4606)
* cuda : improve cuda pool efficiency using virtual memory
* fix mixtral
* fix cmake build
* check for vmm support, disable for hip
ggml-ci
* fix hip build
* clarify granularity
* move all caps to g_device_caps
* refactor error checking
* add cuda_pool_alloc, refactor most pool allocations
ggml-ci
* fix hip build
* CUBLAS_TF32_TENSOR_OP_MATH is not a macro
* more hip crap
* llama : fix msvc warnings
* ggml : fix msvc warnings
* minor
* minor
* cuda : fallback to CPU on host buffer alloc fail
* Update ggml-cuda.cu
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
* Update ggml-cuda.cu
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
* ensure allocations are always aligned
* act_size -> actual_size
---------
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
Diffstat (limited to 'gguf-py/gguf/tensor_mapping.py')
0 files changed, 0 insertions, 0 deletions