diff options
| author | BarfingLemurs <128182951+BarfingLemurs@users.noreply.github.com> | 2023-09-27 11:30:36 -0400 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-09-27 18:30:36 +0300 |
| commit | ffe88a36a913e5792aa383f0726bdbcf632e7191 (patch) | |
| tree | 1674d754253bb233833f20245719e48c5b6c74c4 /examples/perplexity/README.md | |
| parent | 99115f3fa654b593099c6719ad30e3f54ce231e1 (diff) | |
readme : add some recent perplexity and bpw measurements to READMES, link for k-quants (#3340)
* Update README.md
* Update README.md
* Update README.md with k-quants bpw measurements
Diffstat (limited to 'examples/perplexity/README.md')
| -rw-r--r-- | examples/perplexity/README.md | 18 |
1 files changed, 18 insertions, 0 deletions
diff --git a/examples/perplexity/README.md b/examples/perplexity/README.md index eacfb17c..50e1af01 100644 --- a/examples/perplexity/README.md +++ b/examples/perplexity/README.md @@ -1,3 +1,21 @@ # perplexity TODO + +## Llama 2 70B Scorechart +Quantization | Model size (GiB) | Perplexity | Delta to fp16 +-- | -- | -- | -- +Q4_0 | 36.20 | 3.5550 | 3.61% +Q4_1 | 40.20 | 3.5125 | 2.37% +Q5_0 | 44.20 | 3.4744 | 1.26% +Q2_K | 27.27 | 3.7339 | 8.82% +Q3_K_S | 27.86 | 3.7019 | 7.89% +Q3_K_M | 30.83 | 3.5932 | 4.72% +Q3_K_L | 33.67 | 3.5617 | 3.80% +Q4_K_S | 36.39 | 3.4852 | 1.57% +Q4_K_M | 38.54 | 3.4725 | 1.20% +Q5_K_S | 44.20 | 3.4483 | 0.50% +Q5_K_M | 45.41 | 3.4451 | 0.40% +Q6_K | 52.70 | 3.4367 | 0.16% +fp16 | 128.5 | 3.4313 | - + |