diff options
| author | postmasters <namnguyen@google.com> | 2024-01-02 03:51:28 -0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-01-02 13:51:28 +0200 |
| commit | 83e633c27efdf0eb0ba54249e784b0ea760b1007 (patch) | |
| tree | 30711187d9551899c546f9181f00456481873679 /examples/perplexity/CMakeLists.txt | |
| parent | 32866c5edde402f42ff4233bb89dcfcede34fd22 (diff) | |
llama : differentiate the KV dims in the attention (#4657)
* Add n_key_dim and n_value_dim
Some models use values that are not derived from `n_embd`.
Also remove `n_embd_head` and `n_embd_gqa` because it is not clear
which "head" is referred to (key or value).
Fix issue #4648.
* Fix `llm_build_kqv` to use `n_value_gqa`
* Rebase
* Rename variables
* Fix llm_build_kqv to be more generic wrt n_embd_head_k
* Update default values for n_embd_head_k and n_embd_head_v
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* Fix llm_load_tensors: the asserts were not backcompat
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Diffstat (limited to 'examples/perplexity/CMakeLists.txt')
0 files changed, 0 insertions, 0 deletions