diff options
| author | jon-chuang <9093549+jon-chuang@users.noreply.github.com> | 2023-04-30 14:41:35 -0400 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-04-30 21:41:35 +0300 |
| commit | a5d30b1f53677cb50791fec41c43e93274347303 (patch) | |
| tree | 48cc90778591816a0aeac1e1782ae190c7e1a79d /examples/common.h | |
| parent | 76a884920aa1d2fc0dc7a7ac12dfc5ec5816377c (diff) | |
common : better default number of threads (#934)
* commit
* fix
* try-catch
* apply code review
* improve
* improve
* add macos headers
* done
* remove color
* fix windows
* minor
* fix
* Apply suggestions from code review
Co-authored-by: DannyDaemonic <DannyDaemonic@gmail.com>
* remove
* minor
* minor
---------
Co-authored-by: jon-chuang <jon-chuang@users.noreply.github.com>
Co-authored-by: DannyDaemonic <DannyDaemonic@gmail.com>
Diffstat (limited to 'examples/common.h')
| -rw-r--r-- | examples/common.h | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/examples/common.h b/examples/common.h index fce1d42a..627696e3 100644 --- a/examples/common.h +++ b/examples/common.h @@ -13,11 +13,12 @@ // // CLI argument parsing // +int32_t get_num_physical_cores(); struct gpt_params { int32_t seed = -1; // RNG seed - int32_t n_threads = std::min(4, (int32_t) std::thread::hardware_concurrency()); - int32_t n_predict = -1; // new tokens to predict + int32_t n_threads = get_num_physical_cores(); + int32_t n_predict = -1; // new tokens to predict int32_t n_parts = -1; // amount of model parts (-1 = determine from model dimensions) int32_t n_ctx = 512; // context size int32_t n_batch = 512; // batch size for prompt processing (must be >=32 to use BLAS) |