Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-11-25 17:29:22 +01:00
Code Issues Packages Projects Releases Wiki Activity

Update ExLlama.md (#2729)

Browse Source 
Add details for configuring exllama
This commit is contained in:
Jonathan Yankovich 2023-06-16 21:46:25 -05:00 committed by GitHub
parent b27f83c0e9
commit a1ca1c04a1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/ExLlama.md
View File

@ -2,7 +2,7 @@
## About ## About
ExLlama is an extremely optimized GPTQ backend for LLaMA models. It features much lower VRAM usage and much higher speeds due to not relying on unoptimized transformers code. ExLlama is an extremely optimized GPTQ backend ("loader") for LLaMA models. It features much lower VRAM usage and much higher speeds due to not relying on unoptimized transformers code.
## Installation: ## Installation:
@ -15,3 +15,7 @@ git clone https://github.com/turboderp/exllama
``` ```
2) Follow the remaining set up instructions in the official README: https://github.com/turboderp/exllama#exllama 2) Follow the remaining set up instructions in the official README: https://github.com/turboderp/exllama#exllama
3) Configure text-generation-webui to use exllama via the UI or command line:
- In the "Model" tab, set "Loader" to "exllama"
- Specify `--loader exllama` on the command line