diff options
| author | kalomaze <66376113+kalomaze@users.noreply.github.com> | 2023-10-31 14:44:49 -0500 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-10-31 20:44:49 +0100 |
| commit | 238657db2364cfb728c694470a4a81702afea760 (patch) | |
| tree | 8b870a0600d1a2de4d9efe7981c24164357f5552 /examples | |
| parent | 07178c98e1b61a5e2af39d347add12e7eb9e08e1 (diff) | |
samplers : Min-P sampler implementation [alternative to Top P/Top K] (#3841)
* Introduce the new Min-P sampler by @kalomaze
The Min-P sampling method was designed as an alternative to Top-P, and aims to ensure a balance of quality and variety. The parameter *p* represents the minimum probability for a token to be considered, relative to the probability of the most likely token.
* Min-P enabled and set to 0.05 default
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>
Diffstat (limited to 'examples')
| -rw-r--r-- | examples/main/README.md | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/examples/main/README.md b/examples/main/README.md index a9561c38..a3428b48 100644 --- a/examples/main/README.md +++ b/examples/main/README.md @@ -208,6 +208,14 @@ Top-p sampling, also known as nucleus sampling, is another text generation metho Example usage: `--top-p 0.95` +### Min P Sampling + +- `--min-p N`: Sets a minimum base probability threshold for token selection (default: 0.05). + +The Min-P sampling method was designed as an alternative to Top-P, and aims to ensure a balance of quality and variety. The parameter *p* represents the minimum probability for a token to be considered, relative to the probability of the most likely token. For example, with *p*=0.05 and the most likely token having a probability of 0.9, logits with a value less than 0.045 are filtered out. + +Example usage: `--min-p 0.05` + ### Tail Free Sampling (TFS) - `--tfs N`: Enable tail free sampling with parameter z (default: 1.0, 1.0 = disabled). |