mirror of
https://github.com/oobabooga/text-generation-webui.git
synced 2024-12-04 21:10:30 +01:00
Updated Home (markdown)
parent
3616d2ecdc
commit
9ad712abe9
28
Home.md
28
Home.md
@ -8,9 +8,9 @@
|
|||||||
|-----|-----|-----|-----|-----|
|
|-----|-----|-----|-----|-----|
|
||||||
| Transformers | ✅ | ✅* | ✅ | ✅ | ✅ |
|
| Transformers | ✅ | ✅* | ✅ | ✅ | ✅ |
|
||||||
| ExLlama_HF | ✅ | ❌ | ❌ | ✅ | ✅ |
|
| ExLlama_HF | ✅ | ❌ | ❌ | ✅ | ✅ |
|
||||||
| ExLlamav2_HF | ✅ | ❌ | ❌ | ✅ | ✅ |
|
| ExLlamav2_HF | ❌ | ❌ | ❌ | ✅ | ✅ |
|
||||||
| ExLlama | ✅ | ❌ | ❌ | ❌ | ✅ |
|
| ExLlama | ✅ | ❌ | ❌ | ❌ | ✅ |
|
||||||
| ExLlamav2 | ✅ | ❌ | ❌ | ❌ | ❌ |
|
| ExLlamav2 | ❌ | ❌ | ❌ | ❌ | ❌ |
|
||||||
| AutoGPTQ | ✅ | ❌ | ✅ | ✅ | ✅ |
|
| AutoGPTQ | ✅ | ❌ | ✅ | ✅ | ✅ |
|
||||||
| GPTQ-for-LLaMa | ✅** | ✅** | ✅ | ✅ | ✅ |
|
| GPTQ-for-LLaMa | ✅** | ✅** | ✅ | ✅ | ✅ |
|
||||||
| llama.cpp | ❌ | ❌ | ❌ | ❌ | ❌ |
|
| llama.cpp | ❌ | ❌ | ❌ | ❌ | ❌ |
|
||||||
@ -242,4 +242,26 @@ Four tabs can be found:
|
|||||||
|
|
||||||
### Notebook tab
|
### Notebook tab
|
||||||
|
|
||||||
Precisely the same thing as the Default tab, with the difference that the output appears in the same text box as the input.
|
Precisely the same thing as the Default tab, with the difference that the output appears in the same text box as the input.
|
||||||
|
|
||||||
|
## Using LoRAs with GPTQ-for-LLaMa
|
||||||
|
|
||||||
|
This requires using a monkey patch that is supported by this web UI: https://github.com/johnsmith0031/alpaca_lora_4bit
|
||||||
|
|
||||||
|
To use it:
|
||||||
|
|
||||||
|
1. Install alpaca_lora_4bit using pip
|
||||||
|
|
||||||
|
```
|
||||||
|
git clone https://github.com/johnsmith0031/alpaca_lora_4bit.git
|
||||||
|
cd alpaca_lora_4bit
|
||||||
|
git fetch origin winglian-setup_pip
|
||||||
|
git checkout winglian-setup_pip
|
||||||
|
pip install .
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Start the UI with the `--monkey-patch` flag:
|
||||||
|
|
||||||
|
```
|
||||||
|
python server.py --model llama-7b-4bit-128g --listen --lora tloen_alpaca-lora-7b --monkey-patch
|
||||||
|
```
|
Loading…
Reference in New Issue
Block a user