diff options
| author | Behnam M <58621210+ibehnam@users.noreply.github.com> | 2024-01-11 02:12:05 -0500 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-01-11 09:12:05 +0200 |
| commit | 7a9f75c38b5e62fe27b8a5a3ed823b4a3714024b (patch) | |
| tree | f4a60e38d886f8d72b051541e6d27823d4d35a11 /examples/server | |
| parent | 5c1980d8d4c4e0c0af77359f81cc44d90b3f250b (diff) | |
server : update readme to document the new `/health` endpoint (#4866)
* added /health endpoint to the server
* added comments on the additional /health endpoint
* Better handling of server state
When the model is being loaded, the server state is `LOADING_MODEL`. If model-loading fails, the server state becomes `ERROR`, otherwise it becomes `READY`. The `/health` endpoint provides more granular messages now according to the server_state value.
* initialized server_state
* fixed a typo
* starting http server before initializing the model
* Update server.cpp
* Update server.cpp
* fixes
* fixes
* fixes
* made ServerState atomic and turned two-line spaces into one-line
* updated `server` readme to document the `/health` endpoint too
Diffstat (limited to 'examples/server')
| -rw-r--r-- | examples/server/README.md | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/examples/server/README.md b/examples/server/README.md index d85a14f8..dc27e72b 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -110,6 +110,10 @@ node index.js ``` ## API Endpoints +- **GET** `/health`: Returns the current state of the server: + - `{"status": "loading model"}` if the model is still being loaded. + - `{"status": "error"}` if the model failed to load. + - `{"status": "ok"}` if the model is successfully loaded and the server is ready for further requests mentioned below. - **POST** `/completion`: Given a `prompt`, it returns the predicted completion. |