diff options
| author | fairydreaming <166155368+fairydreaming@users.noreply.github.com> | 2024-05-28 17:07:05 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-28 17:07:05 +0200 |
| commit | ee3dff6b8e39bb8c1cdea1782a7b95ef0118f970 (patch) | |
| tree | 28eaea501c6c929f98442cc451eb14b380c02de6 /gguf-py/examples/writer.py | |
| parent | edc29433fa08b4e5aeb67649a29fc7713af13d04 (diff) | |
Add support for DeepseekV2ForCausalLM (#7519)
* common : increase max number of experts to 160
* common : add tensors ATTN_Q_A, ATTN_Q_A_NORM, ATTN_Q_B, ATTN_KV_A_MQA, ATTN_KV_A_NORM, ATTN_KV_B needed by DeepSeek-V2 MLA (multi-head latent attention) architecture
* common : add model header parameters: leading_dense_block_count, expert_feed_forward_length, expert_shared_count, expert_weights_scale, attention.q_lora_rank, attention.kv_lora_rank, rope.scaling.yarn_log_multiplier
* convert-hf : add model conversion support for DeepseekV2ForCausalLM
* llama : add model types for DeepSeek-V2 and DeepSeek-V2-Lite models
* llama : add two new llm_build_moe_ffn() arguments: scale_w (whether to scale weights of selected MoE experts) and w_scale (numerical value of the scaling factor)
* llama : add inference support for LLM_ARCH_DEEPSEEK2
---------
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
Diffstat (limited to 'gguf-py/examples/writer.py')
0 files changed, 0 insertions, 0 deletions