diff options
| author | Jorge A <161275481+jorgealias@users.noreply.github.com> | 2024-02-28 01:39:15 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-02-28 10:39:15 +0200 |
| commit | efc72253f7987ed7bdc8bde9d9fa5c7cac2f6292 (patch) | |
| tree | 9f208051c3b76fa9817b748e9d2b805b439d75a5 /examples/server/tests/features/parallel.feature | |
| parent | 7c4263d4261d6ee6f0539d53eb9e1b4d120ba8af (diff) | |
server : add "/chat/completions" alias for "/v1/...` (#5722)
* Add "/chat/completions" as alias for "/v1/chat/completions"
* merge to upstream master
* minor : fix trailing whitespace
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Diffstat (limited to 'examples/server/tests/features/parallel.feature')
| -rw-r--r-- | examples/server/tests/features/parallel.feature | 22 |
1 files changed, 22 insertions, 0 deletions
diff --git a/examples/server/tests/features/parallel.feature b/examples/server/tests/features/parallel.feature index c85f9de1..5f895cf9 100644 --- a/examples/server/tests/features/parallel.feature +++ b/examples/server/tests/features/parallel.feature @@ -54,6 +54,28 @@ Feature: Parallel | disabled | 128 | | enabled | 64 | + Scenario Outline: Multi users OAI completions compatibility no v1 + Given a system prompt You are a writer. + And a model tinyllama-2 + Given a prompt: + """ + Write a very long book. + """ + And a prompt: + """ + Write another a poem. + """ + And <n_predict> max tokens to predict + And streaming is <streaming> + Given concurrent OAI completions requests no v1 + Then the server is busy + Then the server is idle + Then all prompts are predicted with <n_predict> tokens + Examples: + | streaming | n_predict | + | disabled | 128 | + | enabled | 64 | + Scenario: Multi users with total number of tokens to predict exceeds the KV Cache size #3969 Given a prompt: """ |