Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-11-22 16:17:57 +01:00
Code Issues Packages Projects Releases Wiki Activity

Update llama.cpp.md

Browse Source
This commit is contained in:
oobabooga 2023-07-25 15:33:16 -07:00
parent 863d2f118f
commit d3abe7caa8
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/llama.cpp.md
View File

@ -24,7 +24,7 @@ Enabled with the `--n-gpu-layers` parameter.
* If you have enough VRAM, use a high number like `--n-gpu-layers 1000` to offload all layers to the GPU. * If you have enough VRAM, use a high number like `--n-gpu-layers 1000` to offload all layers to the GPU.
* Otherwise, start with a low number like `--n-gpu-layers 10` and then gradually increase it until you run out of memory. * Otherwise, start with a low number like `--n-gpu-layers 10` and then gradually increase it until you run out of memory.
This feature works out of the box for NVIDIA GPUs. For other GPUs, you need to uninstall `llama-cpp-python` with This feature works out of the box for NVIDIA GPUs on Linux (amd64) or Windows. For other GPUs, you need to uninstall `llama-cpp-python` with
``` ```
pip uninstall -y llama-cpp-python pip uninstall -y llama-cpp-python