diff options
| author | goerch <jhr.walter@t-online.de> | 2023-09-13 15:19:44 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-09-13 16:19:44 +0300 |
| commit | 71ca2fad7d6c0ef95ef9944fb3a1a843e481f314 (patch) | |
| tree | 21bc810807b527d5892e8184d5f1dae0b184e923 /tests/test-tokenizer-0-llama.cpp | |
| parent | 1b6c650d16048d6427dd502a9627e72837265844 (diff) | |
whisper : tokenizer fix + re-enable tokenizer test for LLaMa (#3096)
* Fix für #2721
* Reenable tokenizer test for LLaMa
* Add `console.cpp` dependency
* Fix dependency to `common`
* Fixing wrong fix.
* Make console usage platform specific
Work on compiler warnings.
* Adapting makefile
* Remove trailing whitespace
* Adapting the other parts of the makefile
* Fix typo.
Diffstat (limited to 'tests/test-tokenizer-0-llama.cpp')
| -rw-r--r-- | tests/test-tokenizer-0-llama.cpp | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/tests/test-tokenizer-0-llama.cpp b/tests/test-tokenizer-0-llama.cpp index 8630742c..edbd86f8 100644 --- a/tests/test-tokenizer-0-llama.cpp +++ b/tests/test-tokenizer-0-llama.cpp @@ -1,5 +1,6 @@ #include "llama.h" #include "common.h" +#include "console.h" #include <cstdio> #include <string> @@ -89,6 +90,12 @@ int main(int argc, char **argv) { return 2; } +#ifdef _WIN32 + // We need this for unicode console support + console::init(false, false); + atexit([]() { console::cleanup(); }); +#endif + bool success = true; for (const auto & test_kv : k_tests()) { |