diff options
| author | Robey Holderith <robey@flaminglunchbox.net> | 2024-02-18 11:11:16 -0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-02-18 21:11:16 +0200 |
| commit | 5ee99c32f5e47c8d32634eff9a47fb32a24c276b (patch) | |
| tree | b96e45b36fe325a926025f94655a885428c84f47 /examples/server/README.md | |
| parent | c145f8a132b2fe1d1e65987faddbd9a40bef7a12 (diff) | |
common, server : surface min_keep as its own parameter (#5567)
* Feature - surface min_keep as its own parameter
* Updated README with min_keep param
Diffstat (limited to 'examples/server/README.md')
| -rw-r--r-- | examples/server/README.md | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/examples/server/README.md b/examples/server/README.md index ac5133d2..809e2d37 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -199,6 +199,8 @@ node index.js `n_probs`: If greater than 0, the response also contains the probabilities of top N tokens for each generated token (default: 0) + `min_keep`: If greater than 0, force samplers to return N possible tokens at minimum (default: 0) + `image_data`: An array of objects to hold base64-encoded image `data` and its `id`s to be reference in `prompt`. You can determine the place of the image in the prompt as in the following: `USER:[img-12]Describe the image in detail.\nASSISTANT:`. In this case, `[img-12]` will be replaced by the embeddings of the image with id `12` in the following `image_data` array: `{..., "image_data": [{"data": "<BASE64_STRING>", "id": 12}]}`. Use `image_data` only with multimodal models, e.g., LLaVA. `slot_id`: Assign the completion task to an specific slot. If is -1 the task will be assigned to a Idle slot (default: -1) |