diff options
| author | agray3 <agray3@users.noreply.github.com> | 2024-05-15 14:44:49 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-15 15:44:49 +0200 |
| commit | dc020985b8755dd6aa93a2f002f43c3ede808cce (patch) | |
| tree | a4be81a8ce9f08fbafbc92c3e38ee892192bfe91 /examples/main-cmake-pkg | |
| parent | 344f9126cc0d15891fde9472fe40b8572628ad7d (diff) | |
Avoid unnecessarily disabling CUDA graphs (#7302)
As discussed in PR #6766, CUDA graphs were being disabled in the presence of long prompts.
This fixes the issue by avoiding the consective update counter from incrementing unnecessarily
for tokens in which cuda graphs are disabled due to batch size > 1.
Diffstat (limited to 'examples/main-cmake-pkg')
0 files changed, 0 insertions, 0 deletions