diff options
| author | Minsoo Cheong <54794500+mscheong01@users.noreply.github.com> | 2024-03-05 03:24:00 +0900 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-03-04 20:24:00 +0200 |
| commit | 6d341ab6c53cd51f2921d986d0090cc8b049b39a (patch) | |
| tree | f212b497e210c8c73fe52369f6bc81297c7b1dab /ggml-backend-impl.h | |
| parent | 4ffcdce2ff877ebb683cd217ea38faf20faa5ffe (diff) | |
speculative : implement stochastic speculative sampling (#5625)
* (WIP) Implement stochastic speculative decoding
* sample from residual distribution on draft accept failure
* fix #5657: force greedy sampling with probs when temp is 0
* remove p_accept parameter
* fix style
* remove unused variables
* add srand() in speculative.cpp
* replace use of rand() with mt19937 sampling
* fixes based on review (@JohannesGaessler)
* fix r random generation
* randomly select next sequence to verify + fix bug in memory freeing
* fix bug in active_seqs sync
* fix uniform int distribution initialization
* remove warnings from comparison between int and size_t
* check grammar in `llama_sample_probability_distribution_impl`
* remove malloc code by utilizing vectors
* add PR link to README
Diffstat (limited to 'ggml-backend-impl.h')
0 files changed, 0 insertions, 0 deletions