diff options
| author | agray3 <agray3@users.noreply.github.com> | 2024-10-20 07:36:16 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-10-20 08:36:16 +0200 |
| commit | f2d315b46f7aacc7df4b86bd8acba387b30e11ca (patch) | |
| tree | 05df4e87b268b03a68f18907c9ee7492f9159518 /ggml/src/ggml.c | |
| parent | afbf2ef3e263e5bee3150cf9c422039aa406f10d (diff) | |
Avoid rebuild of GGML graph for each token (#98)
Introduces caching of GGML graph to avoid unnecessary full rebuild between each token.
KV cache parameters, which change with each token, are updated directly in cached GGML
graph. Can be disabled with GGML_DISABLE_GRAPH_CACHING environment variable.
Diffstat (limited to 'ggml/src/ggml.c')
0 files changed, 0 insertions, 0 deletions