diff options
| author | Lee Drake <b.lee.drake@gmail.com> | 2023-09-21 13:00:24 -0600 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-09-21 21:00:24 +0200 |
| commit | bc9d3e3971e5607a10ff4c24e39568ce1ac87271 (patch) | |
| tree | 90d1dd14a081da0507a5aece44cfb95143b662ee | |
| parent | 36b904e20003017f50108ae68359ef87a192dae2 (diff) | |
Update README.md (#3289)
* Update README.md
* Update README.md
Co-authored-by: slaren <slarengh@gmail.com>
---------
Co-authored-by: slaren <slarengh@gmail.com>
| -rw-r--r-- | README.md | 4 |
1 files changed, 4 insertions, 0 deletions
@@ -557,6 +557,10 @@ python3 convert.py models/7B/ # quantize the model to 4-bits (using q4_0 method) ./quantize ./models/7B/ggml-model-f16.gguf ./models/7B/ggml-model-q4_0.gguf q4_0 +# update the gguf filetype to current if older version is unsupported by another application +./quantize ./models/7B/ggml-model-q4_0.gguf ./models/7B/ggml-model-q4_0-v2.gguf COPY + + # run the inference ./main -m ./models/7B/ggml-model-q4_0.gguf -n 128 ``` |