diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2024-03-09 15:47:47 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-03-09 15:47:47 +0200 |
| commit | 97c09585d65a95864773b4d25d66d0f708baf38d (patch) | |
| tree | 0906f33a9f6fdbba48d831368e83b14f7f0f6561 | |
| parent | fb215c3832236fec7380c4fb618bd7154cb196ef (diff) | |
server : clarify some items in the readme (#5957)
* server : clarify some items in the readme
* server : fix typo
| -rw-r--r-- | examples/server/README.md | 8 |
1 files changed, 6 insertions, 2 deletions
diff --git a/examples/server/README.md b/examples/server/README.md index 3abb1abe..23606b32 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -195,7 +195,11 @@ node index.js *Options:* - `prompt`: Provide the prompt for this completion as a string or as an array of strings or numbers representing tokens. Internally, the prompt is compared to the previous completion and only the "unseen" suffix is evaluated. If the prompt is a string or an array with the first element given as a string, a `bos` token is inserted in the front like `main` does. + `prompt`: Provide the prompt for this completion as a string or as an array of strings or numbers representing tokens. Internally, if `cache_prompt` is `true`, the prompt is compared to the previous completion and only the "unseen" suffix is evaluated. A `BOS` token is inserted at the start, if all of the following conditions are true: + + - The prompt is a string or an array with the first element given as a string + - The model's `tokenizer.ggml.add_bos_token` metadata is `true` + - The system prompt is empty `temperature`: Adjust the randomness of the generated text (default: 0.8). @@ -308,7 +312,7 @@ Notice that each `probs` is an array of length `n_probs`. `content`: Set the text to tokenize. - Note that the special `BOS` token is not added in front of the text and also a space character is not inserted automatically as it is for `/completion`. + Note that a special `BOS` token is never inserted. - **POST** `/detokenize`: Convert tokens to text. |