diff options
| author | Daniel Bevenius <daniel.bevenius@gmail.com> | 2024-04-12 14:11:46 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-04-12 15:11:46 +0300 |
| commit | 4cc120c7443cf9dab898736f3c3b45dc8f14672b (patch) | |
| tree | e2364cc54ae6a3a836476565d117d12ea67eb9c0 | |
| parent | 24ee66ed0d908d156bd0d1747b63a636a495cd7a (diff) | |
infill : add download instructions for model (#6626)
* infill : add download instructions for model
This commit adds instructions on how to download a CodeLlama model
using the `hf.sh` script. This will download the model and place it
in the `models` directory which is the same model use later by the
infill example.
Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
* squash! infill : add download instructions for model
Clarify the reason for using CodeLlama.
Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
---------
Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
| -rw-r--r-- | examples/infill/README.md | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/examples/infill/README.md b/examples/infill/README.md index 8c97f719..6b076c83 100644 --- a/examples/infill/README.md +++ b/examples/infill/README.md @@ -36,6 +36,11 @@ The `infill` program offers a seamless way to interact with LLaMA models, allowi ### Example +Download a model that supports infill, for example CodeLlama: +```console +scripts/hf.sh --repo TheBloke/CodeLlama-13B-GGUF --file codellama-13b.Q5_K_S.gguf --outdir models +``` + ```bash ./infill -t 10 -ngl 0 -m models/codellama-13b.Q5_K_S.gguf -c 4096 --temp 0.7 --repeat_penalty 1.1 -n 20 --in-prefix "def helloworld():\n print(\"hell" --in-suffix "\n print(\"goodbye world\")\n " ``` |