diff options
| author | Daniel Bevenius <daniel.bevenius@gmail.com> | 2024-01-02 10:16:55 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-01-02 10:16:55 +0100 |
| commit | 775ac8712a7b42cfead2585f42cec0dfd56644ab (patch) | |
| tree | 6e7382867835b143100faccdf151867fd3053555 /examples | |
| parent | 58ba655af054715c0516ee270ad028ad9e74f357 (diff) | |
finetune: fix typo in README.md (#4733)
Signed-off-by: Daniel Bevenius <daniel.bevenius@gmail.com>
Diffstat (limited to 'examples')
| -rw-r--r-- | examples/finetune/README.md | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/examples/finetune/README.md b/examples/finetune/README.md index a2a2c128..a884706c 100644 --- a/examples/finetune/README.md +++ b/examples/finetune/README.md @@ -61,7 +61,7 @@ For example to apply 40% of the 'shakespeare' LORA adapter, 80% of the 'bible' L --lora lora-open-llama-3b-v2-q8_0-yet-another-one-LATEST.bin ``` -The scale numbers don't need to add up to one, and you can also use numbers greater than 1 to further increase the influence of an adapter. But making the values to big will sometimes result in worse output. Play around to find good values. +The scale numbers don't need to add up to one, and you can also use numbers greater than 1 to further increase the influence of an adapter. But making the values too big will sometimes result in worse output. Play around to find good values. Gradient checkpointing reduces the memory requirements by ~50% but increases the runtime. If you have enough RAM, you can make finetuning a bit faster by disabling checkpointing with `--no-checkpointing`. |