summaryrefslogtreecommitdiff
path: root/gguf-py/gguf/tensor_mapping.py
diff options
context:
space:
mode:
authorRick G <26732651+TheFlipbook@users.noreply.github.com>2024-04-08 06:02:30 -0700
committerGitHub <noreply@github.com>2024-04-08 16:02:30 +0300
commite3c337d87ca650972105a51c6ce302dd236c07ad (patch)
treee91b25c531cc5f508e64309071f4a904a0d27189 /gguf-py/gguf/tensor_mapping.py
parentbeea6e1b16e783a0886e78dec01002a8c00db24d (diff)
llama : support negative ith in llama_get_ API (#6519)
* llama_sampling_sample with default args is more naively usable * Batches populated by either llama_batch_get_one or llama_batch_add work with default args * Previously get_one could use the default argument * Previously add should usually have used the last index where logits[idx] == true * This hopefully encourages the use of llama_batch_add * By giving expected results when using default arguments. * Adds "negative indexing" feature to llama_get_logits_ith and llama_get_embeddings_ith * Believed to work with any currently well behaved program * Default arg now works for both cases (previously would give strange results for add case) * Any non-negative number is unaffected and behaves as previously * Negative arguments were previously invalid. * Implemented as a special case of indexing as suggested by @compilade in https://github.com/ggerganov/llama.cpp/pull/6519 * Fixed mismatch type errors * cited in macOS CI tests * Missed in original updates based on PR feedback in https://github.com/ggerganov/llama.cpp/pull/6519
Diffstat (limited to 'gguf-py/gguf/tensor_mapping.py')
0 files changed, 0 insertions, 0 deletions