From b804b1ef77351d2a11be945462c6c251710476cb Mon Sep 17 00:00:00 2001 From: Pierrick Hymbert Date: Thu, 11 Apr 2024 14:51:07 +0200 Subject: eval-callback: Example how to use eval callback for debugging (#6576) * gguf-debug: Example how to use ggml callback for debugging * gguf-debug: no mutex, verify type, fix stride. * llama: cv eval: move cb eval field in common gpt_params * ggml_debug: use common gpt_params to pass cb eval. Fix get tensor SIGV random. * ggml_debug: ci: add tests * ggml_debug: EOL in CMakeLists.txt * ggml_debug: Remove unused param n_batch, no batching here * ggml_debug: fix trailing spaces * ggml_debug: fix trailing spaces * common: fix cb_eval and user data not initialized * ci: build revert label * ggml_debug: add main test label * doc: add a model: add a link to ggml-debug * ggml-debug: add to make toolchain * ggml-debug: tests add the main label * ggml-debug: ci add test curl label * common: allow the warmup to be disabled in llama_init_from_gpt_params * ci: add curl test * ggml-debug: better tensor type support * gitignore : ggml-debug * ggml-debug: printing also the sum of each tensor * ggml-debug: remove block size * eval-callback: renamed from ggml-debug * eval-callback: fix make toolchain --------- Co-authored-by: slaren Co-authored-by: Georgi Gerganov --- docs/HOWTO-add-model.md | 2 ++ 1 file changed, 2 insertions(+) (limited to 'docs') diff --git a/docs/HOWTO-add-model.md b/docs/HOWTO-add-model.md index 3581f3e6..a56b7834 100644 --- a/docs/HOWTO-add-model.md +++ b/docs/HOWTO-add-model.md @@ -100,6 +100,8 @@ Have a look to existing implementation like `build_llama`, `build_dbrx` or `buil When implementing a new graph, please note that the underlying `ggml` backends might not support them all, support of missing backend operations can be added in another PR. +Note: to debug the inference graph: you can use [eval-callback](../examples/eval-callback). + ## GGUF specification https://github.com/ggerganov/ggml/blob/master/docs/gguf.md -- cgit v1.2.3