diff options
| author | Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com> | 2023-12-05 10:19:18 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-12-05 19:19:18 +0200 |
| commit | 5aa365d88fdb8fdd430ef3fc141c7a5fd37c3502 (patch) | |
| tree | b9653918c6b5cd09b5a981927b143d3011456acb /common/common.h | |
| parent | 52c8bc3cf312e1caf02d37bfb9d9d865cbe33594 (diff) | |
llama : allow overriding GGUF metadata when loading model (#4092)
* feat: Allow overriding GGUF metadata when loading model
* Fix the one time GCC is stricter than clang about something
* Step1
* Refactor... basically everything!
* Nuke obsolete GetArrayLen struct
* simplify std::string specialization
* Various cleanups
Add informational output when overrides are applied
Warn user when an override with the wrong type is specified
* Fix broken logic for parsing bool KV overrides
Fix issue where overrides didn't apply when key missing in GGUF metadata
Resolve merge changes
* llama : rearrange model params
* Update new GET_KEY call
Add note that metadata KV overrides aren't reflected in initial metadata KV info dump
---------
Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Diffstat (limited to 'common/common.h')
| -rw-r--r-- | common/common.h | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/common/common.h b/common/common.h index 534f7b13..02467938 100644 --- a/common/common.h +++ b/common/common.h @@ -86,6 +86,8 @@ struct gpt_params { std::vector<std::string> antiprompt; // string upon seeing which more user input is prompted std::string logdir = ""; // directory in which to save YAML log files + std::vector<llama_model_kv_override> kv_overrides; + // TODO: avoid tuple, use struct std::vector<std::tuple<std::string, float>> lora_adapter; // lora adapter path with user defined scale std::string lora_base = ""; // base model path for the lora adapter |