diff options
| author | Pierrick Hymbert <pierrick.hymbert@gmail.com> | 2024-02-20 08:48:19 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-02-20 09:48:19 +0200 |
| commit | c0a8c6db371cb3e4379900867b948879f5842201 (patch) | |
| tree | bf818b206dca759d3f77bcc2f178c551fb0544a6 /examples/server/README.md | |
| parent | b9111bd209c7b11b0592450a6ed2e0ca545b2c84 (diff) | |
server : health endpoint configurable failure on no slot (#5594)
Diffstat (limited to 'examples/server/README.md')
| -rw-r--r-- | examples/server/README.md | 9 |
1 files changed, 5 insertions, 4 deletions
diff --git a/examples/server/README.md b/examples/server/README.md index 809e2d37..f6b9c740 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -134,10 +134,11 @@ node index.js ## API Endpoints - **GET** `/health`: Returns the current state of the server: - - `{"status": "loading model"}` if the model is still being loaded. - - `{"status": "error"}` if the model failed to load. - - `{"status": "ok"}` if the model is successfully loaded and the server is ready for further requests mentioned below. - - `{"status": "no slot available", "slots_idle": 0, "slots_processing": 32}` if no slot are currently available + - 503 -> `{"status": "loading model"}` if the model is still being loaded. + - 500 -> `{"status": "error"}` if the model failed to load. + - 200 -> `{"status": "ok", "slots_idle": 1, "slots_processing": 2 }` if the model is successfully loaded and the server is ready for further requests mentioned below. + - 200 -> `{"status": "no slot available", "slots_idle": 0, "slots_processing": 32}` if no slot are currently available. + - 503 -> `{"status": "no slot available", "slots_idle": 0, "slots_processing": 32}` if the query parameter `fail_on_no_slot` is provided and no slot are currently available. - **POST** `/completion`: Given a `prompt`, it returns the predicted completion. |