diff options
| author | Thireus ☠ <Thireus@users.noreply.github.com> | 2025-07-16 13:11:19 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-07-16 14:11:19 +0200 |
| commit | 4803142300670088f10f61f9ab66d882e93df4b3 (patch) | |
| tree | 6df72beea459bafcd7839804f739483d8c009f4a | |
| parent | 13b2f193723486f46efe34297cf797186ab14bc2 (diff) | |
Bump GGML_MAX_CONTEXTS to allow loading more shards (#611)
* Bump GGML_MAX_CONTEXTS to allow loading more shards
This var prevents more than 64 shards from being loaded - Specifically relevant for large models such as DeepSeek R1.
* https://github.com/ikawrakow/ik_llama.cpp/pull/611#issuecomment-3072175559
| -rw-r--r-- | ggml/include/ggml.h | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/ggml/include/ggml.h b/ggml/include/ggml.h index a85f52b2..644a82b2 100644 --- a/ggml/include/ggml.h +++ b/ggml/include/ggml.h @@ -234,7 +234,12 @@ #define GGML_MAX_DIMS 4 #define GGML_MAX_PARAMS 2048 -#define GGML_MAX_CONTEXTS 64 +#ifndef GGML_MAX_CONTEXTS +// Maximum number of model contexts (e.g., for model shards). +// Increase this value using -DGGML_MAX_CONTEXTS=<value> in CMake +// if you need to load more than 64 model shards. +#define GGML_MAX_CONTEXTS 64 +#endif #define GGML_MAX_SRC 10 #ifndef GGML_MAX_NAME #define GGML_MAX_NAME 64 |