diff options
| author | Johannes Gäßler <johannesg@5d6.de> | 2024-03-23 01:24:36 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-03-23 01:24:36 +0100 |
| commit | 50ccaf5eacb50a2ca378a4ef0dc7aeb45fead652 (patch) | |
| tree | 3ebcfdadf96bb6f3aadd752a1bfe9771ac182d3b /scripts | |
| parent | 56a00f0a2f48a85376f48b5ce77699df781631ae (diff) | |
lookup: complement data from context with general text statistics (#5479)
* lookup: evaluation tools, use corpus/previous gens
* fixup! lookup: evaluation tools, use corpus/previous gens
* fixup! lookup: evaluation tools, use corpus/previous gens
* fixup! lookup: evaluation tools, use corpus/previous gens
* fixup! lookup: evaluation tools, use corpus/previous gens
Diffstat (limited to 'scripts')
| -rwxr-xr-x | scripts/get-wikitext-103.sh | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/scripts/get-wikitext-103.sh b/scripts/get-wikitext-103.sh new file mode 100755 index 00000000..880dd5cb --- /dev/null +++ b/scripts/get-wikitext-103.sh @@ -0,0 +1,10 @@ +#!/bin/bash + +wget https://s3.amazonaws.com/research.metamind.io/wikitext/wikitext-103-raw-v1.zip + +echo "Usage:" +echo "" +echo " ./perplexity -m model.gguf -f wiki.test.raw [other params]" +echo "" + +exit 0 |