diff options
| -rw-r--r-- | examples/main/README.md | 10 | ||||
| -rw-r--r-- | examples/server/README.md | 31 |
2 files changed, 35 insertions, 6 deletions
diff --git a/examples/main/README.md b/examples/main/README.md index dd627456..417432b3 100644 --- a/examples/main/README.md +++ b/examples/main/README.md @@ -241,13 +241,11 @@ Example usage: `--mirostat 2 --mirostat-lr 0.05 --mirostat-ent 3.0` ### XTC Sampling (Exclude Top Choices) -The function of this sampler is conrolled by `--xtc-probability` and `--xtc-threshold`. `--xtc-probability` takes values between -0 and 1 (<=0 turns this sampler off) and defines the probability for randomly invoking the sampler. `--xtc-threshold` -defines the token probability threshold. Tokens with probability greater than this threshold will be excluded from the sampling. -The sampler is turned off for `threshold > 0.5`. +- --xtc-probability p: xtc probability (default: 0.0, maximum: 1.0) => (p*100)% chance to invoke the sampler. +- --xtc-threshold t : xtc threshold (default: 1.0 minimum : 0.0) => (t*100)% is the lower-bound for what probability is needed for a token to be considered a "Top choice", and only the lowest probability top choice is kept, all others are removed. + +The sampler is turned off for `threshold > 0.5`, and `--xtc-probability <=0`. -- --xtc-probability p: xtc probability (default: 0.0 => disabled) -- --xtc-threshold t : xtc threshold (default: 1.0 => disabled) ### Top-n-sigma Sampling diff --git a/examples/server/README.md b/examples/server/README.md index c5a4b3e7..08fc1fa5 100644 --- a/examples/server/README.md +++ b/examples/server/README.md @@ -290,6 +290,37 @@ logging: cmake --build build --config Release -t llama-server ``` +## Web UI + +The project includes a web-based user interface that enables interaction with the model through the `/chat/completions` endpoint. + +The web UI is developed using: +- `vue` framework for frontend development +- `tailwindcss` and `daisyui` for styling +- `vite` for build tooling + +A pre-built version is available as a single HTML file under `/public` directory. + +To build or to run the dev server (with hot reload): + +```sh +# make sure you have nodejs installed +cd examples/server/webui +npm i + +# to run the dev server +npm run dev + +# to build the public/index.html +npm run build +``` + +NOTE: if you are using the vite dev server, you can change the API base URL to llama.cpp. To do that, run this code snippet in browser's console: + +```js +localStorage.setItem('base', 'http://localhost:8080') +``` + ## Quick Start To get started right away, run the following command, making sure to use the correct path for the model you have: |