oobabooga
c5ebcc5f7e
Change the default names ( #518 )
...
* Update shared.py
* Update settings-template.json
2023-03-23 13:36:00 -03:00
oobabooga
7618f3fe8c
Add -gptq-preload for 4-bit offloading ( #460 )
...
This works in a 4GB card now:
```
python server.py --model llama-7b-hf --gptq-bits 4 --gptq-pre-layer 20
```
2023-03-20 16:30:56 -03:00
oobabooga
ddb62470e9
--no-cache and --gpu-memory in MiB for fine VRAM control
2023-03-19 19:21:41 -03:00
oobabooga
a78b6508fc
Make custom LoRAs work by default #385
2023-03-19 12:11:35 -03:00
oobabooga
c753261338
Disable stop_at_newline by default
2023-03-18 10:55:57 -03:00
oobabooga
29fe7b1c74
Remove LoRA tab, move it into the Parameters menu
2023-03-17 11:39:48 -03:00
oobabooga
214dc6868e
Several QoL changes related to LoRA
2023-03-17 11:24:52 -03:00
oobabooga
104293f411
Add LoRA support
2023-03-16 21:31:39 -03:00
oobabooga
4d64a57092
Add Interface mode tab
2023-03-15 23:29:56 -03:00
oobabooga
265ba384b7
Rename a file, add deprecation warning for --load-in-4bit
2023-03-14 07:56:31 -03:00
oobabooga
3da73e409f
Merge branch 'main' into Zerogoki00-opt4-bit
2023-03-14 07:50:36 -03:00
oobabooga
3fb8196e16
Implement "*Is recording a voice message...*" for TTS #303
2023-03-13 22:28:00 -03:00
Ayanami Rei
b6c5c57f2e
remove default value from argument
2023-03-13 22:11:08 +03:00
Ayanami Rei
1b99ed61bc
add argument --gptq-model-type and remove duplicate arguments
2023-03-13 20:01:34 +03:00
oobabooga
66b6971b61
Update README
2023-03-13 12:44:18 -03:00
oobabooga
ddea518e0f
Document --auto-launch
2023-03-13 12:43:33 -03:00
oobabooga
1ddcd4d0ba
Clean up silero_tts
...
This should only be used with --no-stream.
The shared.still_streaming implementation was faulty by design:
output_modifier should never be called when streaming is already over.
2023-03-12 23:42:49 -03:00
oobabooga
d8bea766d7
Merge pull request #192 from xanthousm/main
...
Add text generation stream status to shared module, use for better TTS with auto-play
2023-03-12 13:40:16 -03:00
oobabooga
f3b00dd165
Merge pull request #224 from ItsLogic/llama-bits
...
Allow users to load 2, 3 and 4 bit llama models
2023-03-12 11:23:50 -03:00
oobabooga
65dda28c9d
Rename --llama-bits to --gptq-bits
2023-03-12 11:19:07 -03:00
oobabooga
0ac562bdba
Add a default prompt for OpenAssistant oasst-sft-1-pythia-12b #253
2023-03-12 10:46:16 -03:00
Xan
b3e10e47c0
Fix merge conflict in text_generation
...
- Need to update `shared.still_streaming = False` before the final `yield formatted_outputs`, shifted the position of some yields.
2023-03-12 18:56:35 +11:00
Xan
96c51973f9
--auto-launch and "Is typing..."
...
- Added `--auto-launch` arg to open web UI in the default browser when ready.
- Changed chat.py to display user input immediately and "*Is typing...*" as a temporary reply while generating text. Most noticeable when using `--no-stream`.
2023-03-11 22:50:59 +11:00
Xan
33df4bd91f
Merge remote-tracking branch 'upstream/main'
2023-03-11 22:40:47 +11:00
draff
28fd4fc970
Change wording to be consistent with other args
2023-03-10 23:34:13 +00:00
draff
804486214b
Re-implement --load-in-4bit and update --llama-bits arg description
2023-03-10 23:21:01 +00:00
draff
e6c631aea4
Replace --load-in-4bit with --llama-bits
...
Replaces --load-in-4bit with a more flexible --llama-bits arg to allow for 2 and 3 bit models as well. This commit also fixes a loading issue with .pt files which are not in the root of the models folder
2023-03-10 21:36:45 +00:00
oobabooga
026d60bd34
Remove default preset that didn't do anything
2023-03-10 14:01:02 -03:00
oobabooga
828a524f9a
Add LLaMA 4-bit support
2023-03-09 15:50:26 -03:00
Xan
5648a41a27
Merge branch 'main' of https://github.com/xanthousm/text-generation-webui
2023-03-08 22:08:54 +11:00
Xan
ad6b699503
Better TTS with autoplay
...
- Adds "still_streaming" to shared module for extensions to know if generation is complete
- Changed TTS extension with new options:
- Show text under the audio widget
- Automatically play the audio once text generation finishes
- manage the generated wav files (only keep files for finished generations, optional max file limit)
- [wip] ability to change voice pitch and speed
- added 'tensorboard' to requirements, since python sent "tensorboard not found" errors after a fresh installation.
2023-03-08 22:02:17 +11:00
oobabooga
153dfeb4dd
Add --rwkv-cuda-on parameter, bump rwkv version
2023-03-06 20:12:54 -03:00
oobabooga
bf56b6c1fb
Load settings.json without the need for --settings settings.json
...
This is for setting UI defaults
2023-03-06 10:57:45 -03:00
oobabooga
c33715ad5b
Move towards HF LLaMA implementation
2023-03-05 01:20:31 -03:00
oobabooga
ed8b35efd2
Add --pin-weight parameter for FlexGen
2023-03-04 01:04:02 -03:00
oobabooga
ea5c5eb3da
Add LLaMA support
2023-03-03 14:39:14 -03:00
oobabooga
169209805d
Model-aware prompts and presets
2023-03-02 11:25:04 -03:00
oobabooga
a2a3e8f797
Add --rwkv-strategy parameter
2023-03-01 20:02:48 -03:00
oobabooga
f3da6dcc8f
Merge pull request #149 from oobabooga/RWKV
...
Add RWKV support
2023-03-01 16:57:45 -03:00
oobabooga
a1429d1607
Add default extensions to the settings
2023-02-28 02:20:11 -03:00
oobabooga
ebc64a408c
RWKV support prototype
2023-02-27 23:03:35 -03:00
oobabooga
7c2babfe39
Rename greed to "generation attempts"
2023-02-25 01:42:19 -03:00
oobabooga
7a527a5581
Move "send picture" into an extension
...
I am not proud of how I did it for now.
2023-02-25 00:23:51 -03:00
oobabooga
65326b545a
Move all gradio elements to shared (so that extensions can use them)
2023-02-24 16:46:50 -03:00
oobabooga
f6f792363b
Separate command-line params by spaces instead of commas
2023-02-24 08:55:09 -03:00
luis
5abdc99a7c
gpu-memory arg change
2023-02-23 18:43:55 -05:00
oobabooga
b78561fba6
Minor bug fix
2023-02-23 15:26:41 -03:00
oobabooga
2e86a1ec04
Move chat history into shared module
2023-02-23 15:11:18 -03:00
oobabooga
e46c43afa6
Move some stuff from server.py to modules
2023-02-23 13:42:23 -03:00
oobabooga
1dacd34165
Further refactor
2023-02-23 13:28:30 -03:00
