diff options
| author | Daniel Bevenius <daniel.bevenius@gmail.com> | 2024-05-23 14:29:26 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-23 15:29:26 +0300 |
| commit | 3015851c5ac7334fb544a23a70a284c117b87044 (patch) | |
| tree | a870f840d9b19f561b53eae8048f83283e851cb4 /tests | |
| parent | 55ac3b7aeaf52f19786ed96e885d89521fc0f6c8 (diff) | |
llama : add getters for n_threads/n_threads_batch (#7464)
* llama : add getters for n_threads/n_threads_batch
This commit adds two new functions to the llama API. The functions
can be used to get the number of threads used for generating a single
token and the number of threads used for prompt and batch processing
(multiple tokens).
The motivation for this is that we want to be able to get the number of
threads that the a context is using. The main use case is for a
testing/verification that the number of threads is set correctly.
Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
* squash! llama : add getters for n_threads/n_threads_batch
Rename the getters to llama_n_threads and llama_n_threads_batch.
Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
---------
Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
Diffstat (limited to 'tests')
0 files changed, 0 insertions, 0 deletions