diff options
| author | Johannes Gäßler <johannesg@5d6.de> | 2024-05-07 23:07:58 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-07 23:07:58 +0200 |
| commit | af0a5b616359809ce886ea433acedebb39b12969 (patch) | |
| tree | c9da4c284790e7ddd053c1c5f5e3b636581ee07a /common/sampling.h | |
| parent | b6aa6702030320a3d5fbc2508307af0d7c947e40 (diff) | |
server: fix incorrectly reported token probabilities (#7125)
* server: normalize token probabilities
* fix temperature == 0.0f
Diffstat (limited to 'common/sampling.h')
| -rw-r--r-- | common/sampling.h | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/common/sampling.h b/common/sampling.h index cf7081e3..5b73ecdc 100644 --- a/common/sampling.h +++ b/common/sampling.h @@ -81,6 +81,7 @@ struct llama_sampling_context { // TODO: replace with ring-buffer std::vector<llama_token> prev; std::vector<llama_token_data> cur; + size_t n_considered; std::mt19937 rng; }; |