diff options
author | Volodymyr Vitvitskyi <72226+signalpillar@users.noreply.github.com> | 2023-08-26 14:25:39 +0100 |
---|---|---|
committer | GitHub <noreply@github.com> | 2023-08-26 16:25:39 +0300 |
commit | f305bad11e10ad09e396faed2e37f4f845f5d566 (patch) | |
tree | 8004caa7395bfeef411fbd71f21be0fe15145e48 /examples/perplexity/perplexity.cpp | |
parent | a2ca4e9de9da45ed0bb1c34935d5ec80cebc22d5 (diff) |
flake : build llama.cpp on Intel with nix (#2795)
Problem
-------
`nix build` fails with missing `Accelerate.h`.
Changes
-------
- Fix build of the llama.cpp with nix for Intel: add the same SDK frameworks as
for ARM
- Add `quantize` app to the output of nix flake
- Extend nix devShell with llama-python so we can use convertScript
Testing
-------
Testing the steps with nix:
1. `nix build`
Get the model and then
2. `nix develop` and then `python convert.py models/llama-2-7b.ggmlv3.q4_0.bin`
3. `nix run llama.cpp#quantize -- open_llama_7b/ggml-model-f16.gguf ./models/ggml-model-q4_0.bin 2`
4. `nix run llama.cpp#llama -- -m models/ggml-model-q4_0.bin -p "What is nix?" -n 400 --temp 0.8 -e -t 8`
Co-authored-by: Volodymyr Vitvitskyi <volodymyrvitvitskyi@SamsungPro.local>
Diffstat (limited to 'examples/perplexity/perplexity.cpp')
0 files changed, 0 insertions, 0 deletions