diff options
| author | Pierrick Hymbert <pierrick.hymbert@gmail.com> | 2024-04-06 05:40:47 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-04-06 05:40:47 +0200 |
| commit | 75cd4c77292034ecec587ecb401366f57338f7c0 (patch) | |
| tree | de137718780505410bc75ce219f4bc164961c4fd /examples/main/main.cpp | |
| parent | a8bd14d55717754a1f48313a846a2b16fa998ad2 (diff) | |
ci: bench: support sse and fix prompt processing time / server: add tokens usage in stream OAI response (#6495)
* ci: bench: support sse and fix prompt processing time
server: add tokens usage in stream mode
* ci: bench: README.md EOL
* ci: bench: remove total pp and tg as it is not accurate
* ci: bench: fix case when there is no token generated
* ci: bench: change to the 95 percentile for pp and tg as it is closer to what the server exports in metrics
* ci: bench: fix finish reason rate
Diffstat (limited to 'examples/main/main.cpp')
0 files changed, 0 insertions, 0 deletions