diff options
| author | compilade <113953597+compilade@users.noreply.github.com> | 2024-02-25 13:43:50 -0500 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-02-25 20:43:50 +0200 |
| commit | f7625019c51ca437a5840576d92362cfa710e4a2 (patch) | |
| tree | 6bfc6ccfd3f00857759192a1458a31f1d0b755d9 /examples/server/chat.mjs | |
| parent | abbabc5e51d0d4656b438aec10b7fae9479ef37d (diff) | |
server : fix crash when system prompt is bigger than batch size (#5714)
The system prompt is now decoded in batches.
* server : fix off-by-one n_past when start of prompt matches whole cache
The tokens right after the matching part would otherwise skip a pos value.
Diffstat (limited to 'examples/server/chat.mjs')
0 files changed, 0 insertions, 0 deletions