summaryrefslogtreecommitdiff
path: root/examples/perplexity/perplexity.cpp
diff options
context:
space:
mode:
authorVolodymyr Vitvitskyi <72226+signalpillar@users.noreply.github.com>2023-08-26 14:25:39 +0100
committerGitHub <noreply@github.com>2023-08-26 16:25:39 +0300
commitf305bad11e10ad09e396faed2e37f4f845f5d566 (patch)
tree8004caa7395bfeef411fbd71f21be0fe15145e48 /examples/perplexity/perplexity.cpp
parenta2ca4e9de9da45ed0bb1c34935d5ec80cebc22d5 (diff)
flake : build llama.cpp on Intel with nix (#2795)
Problem ------- `nix build` fails with missing `Accelerate.h`. Changes ------- - Fix build of the llama.cpp with nix for Intel: add the same SDK frameworks as for ARM - Add `quantize` app to the output of nix flake - Extend nix devShell with llama-python so we can use convertScript Testing ------- Testing the steps with nix: 1. `nix build` Get the model and then 2. `nix develop` and then `python convert.py models/llama-2-7b.ggmlv3.q4_0.bin` 3. `nix run llama.cpp#quantize -- open_llama_7b/ggml-model-f16.gguf ./models/ggml-model-q4_0.bin 2` 4. `nix run llama.cpp#llama -- -m models/ggml-model-q4_0.bin -p "What is nix?" -n 400 --temp 0.8 -e -t 8` Co-authored-by: Volodymyr Vitvitskyi <volodymyrvitvitskyi@SamsungPro.local>
Diffstat (limited to 'examples/perplexity/perplexity.cpp')
0 files changed, 0 insertions, 0 deletions