diff options
| author | Pierrick Hymbert <pierrick.hymbert@gmail.com> | 2024-04-26 12:15:30 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-04-26 12:15:30 +0200 |
| commit | 7f5ff558eed0f732af8f25c2ab0645610bdec80c (patch) | |
| tree | a382beee28990c97271ab001de9fa3834f58ed14 /examples/server/bench/bench.py | |
| parent | 9e4e077ec50fde6049b128662c72d37a3c28e34b (diff) | |
server: stop generation at `n_ctx_train` if `n_predict` is not set (#6638)
* server: cap n_predict if not set to n_ctx_train
* server: fix infinite loop
* server: infinite loop, move in process_token
server: infinite loop: set stop limit to true
* minor: spaces
* minor: spaces
* server: include prompt tokens in the EOS limit
Diffstat (limited to 'examples/server/bench/bench.py')
0 files changed, 0 insertions, 0 deletions