diff options
| author | tarcey <cey.tarik@gmail.com> | 2023-11-30 22:40:23 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-11-30 23:40:23 +0200 |
| commit | 954e22858c5cea1dc03e9172d3879402af2b5990 (patch) | |
| tree | c8c80ab70b8953e70770c06ac1af4301d7cf431e /convert.py | |
| parent | e2bd725f4b39bc5c6234858d158e01248f5ab5bd (diff) | |
llama : fix typical sampling (#4261)
Typical sampling was broken because after copying new_candidates into canditates, the "sorted" bool is left at "true", but the new data is no longer sorted according to probability. Patch to set "sorted" to false.
Test: Generating with temp=0.0001 (approx. argmax) should generate the same sequence at typical>=1.0 and typical=0.9999 (approx. disabled, but enters the typical sampling codepath).
Diffstat (limited to 'convert.py')
0 files changed, 0 insertions, 0 deletions