diff options
| author | Pierrick Hymbert <pierrick.hymbert@gmail.com> | 2024-03-23 18:07:00 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-03-23 18:07:00 +0100 |
| commit | f482bb2e4920e544651fb832f2e0bcb4d2ff69ab (patch) | |
| tree | 9fabefd6f3b34aef6bf13a8469c7cdf363cc88cb /common/common.h | |
| parent | 1997577d5e121568ae39f538021733ccd4278c23 (diff) | |
common: llama_load_model_from_url split support (#6192)
* llama: llama_split_prefix fix strncpy does not include string termination
common: llama_load_model_from_url:
- fix header name case sensitive
- support downloading additional split in parallel
- hide password in url
* common: EOL EOF
* common: remove redundant LLAMA_CURL_MAX_PATH_LENGTH definition
* common: change max url max length
* common: minor comment
* server: support HF URL options
* llama: llama_model_loader fix log
* common: use a constant for max url length
* common: clean up curl if file cannot be loaded in gguf
* server: tests: add split tests, and HF options params
* common: move llama_download_hide_password_in_url inside llama_download_file as a lambda
* server: tests: enable back Release test on PR
* spacing
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* spacing
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* spacing
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Diffstat (limited to 'common/common.h')
| -rw-r--r-- | common/common.h | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/common/common.h b/common/common.h index afa4cf6d..a223ecea 100644 --- a/common/common.h +++ b/common/common.h @@ -306,3 +306,10 @@ struct llama_control_vector_load_info { // Load control vectors, scale each by strength, and add them together. // On error, returns {-1, empty} llama_control_vector_data llama_control_vector_load(const std::vector<llama_control_vector_load_info> & load_infos); + +// +// Split utils +// +static const char * const LLM_KV_SPLIT_NO = "split.no"; +static const char * const LLM_KV_SPLIT_COUNT = "split.count"; +static const char * const LLM_KV_SPLIT_TENSORS_COUNT = "split.tensors.count"; |