diff options
| author | Mihai <mihai.chirculescu@yahoo.com> | 2023-10-06 21:39:33 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-10-06 21:39:33 +0300 |
| commit | cb13d73a720c42d1958bff79b6869d77b26b8cea (patch) | |
| tree | 9db31f27dca4775bd110e2188e3d4058860bd374 | |
| parent | 9ca79d5cbbc8d43f2bff951404b6a40ff1ee3788 (diff) | |
server : docs fix default values and add n_probs (#3506)
| -rw-r--r-- | examples/server/README.md | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/examples/server/README.md b/examples/server/README.md index 9ee62d06..8a079ae2 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -114,9 +114,9 @@ node index.js `top_k`: Limit the next token selection to the K most probable tokens (default: 40). - `top_p`: Limit the next token selection to a subset of tokens with a cumulative probability above a threshold P (default: 0.9). + `top_p`: Limit the next token selection to a subset of tokens with a cumulative probability above a threshold P (default: 0.95). - `n_predict`: Set the number of tokens to predict when generating text. **Note:** May exceed the set limit slightly if the last token is a partial multibyte character. When 0, no tokens will be generated but the prompt is evaluated into the cache. (default: 128, -1 = infinity). + `n_predict`: Set the number of tokens to predict when generating text. **Note:** May exceed the set limit slightly if the last token is a partial multibyte character. When 0, no tokens will be generated but the prompt is evaluated into the cache. (default: -1, -1 = infinity). `n_keep`: Specify the number of tokens from the initial prompt to retain when the model resets its internal context. By default, this value is set to 0 (meaning no tokens are kept). Use `-1` to retain all tokens from the initial prompt. @@ -156,6 +156,8 @@ node index.js `logit_bias`: Modify the likelihood of a token appearing in the generated text completion. For example, use `"logit_bias": [[15043,1.0]]` to increase the likelihood of the token 'Hello', or `"logit_bias": [[15043,-1.0]]` to decrease its likelihood. Setting the value to false, `"logit_bias": [[15043,false]]` ensures that the token `Hello` is never produced (default: []). + `n_probs`: If greater than 0, the response also contains the probabilities of top N tokens for each generated token (default: 0) + - **POST** `/tokenize`: Tokenize a given text. *Options:* |