Φφ
483d173d23
Code reuse + indication
...
Now shows the message in the console when unloading weights. Also reload_model() calls unload_model() first to free the memory so that multiple reloads won't overfill it.
2023-03-23 07:06:26 +03:00
Φφ
1917b15275
Unload and reload models on request
2023-03-23 07:06:26 +03:00
oobabooga
29bd41d453
Fix LoRA in CPU mode
2023-03-23 01:05:13 -03:00
oobabooga
eac27f4f55
Make LoRAs work in 16-bit mode
2023-03-23 00:55:33 -03:00
oobabooga
bfa81e105e
Fix FlexGen streaming
2023-03-23 00:22:14 -03:00
oobabooga
7b6f85d327
Fix markdown headers in light mode
2023-03-23 00:13:34 -03:00
oobabooga
de6a09dc7f
Properly separate the original prompt from the reply
2023-03-23 00:12:40 -03:00
oobabooga
d5fc1bead7
Merge pull request #489 from Brawlence/ext-fixes
...
Extensions performance & memory optimisations
2023-03-22 16:10:59 -03:00
oobabooga
bfb1be2820
Minor fix
2023-03-22 16:09:48 -03:00
oobabooga
0abff499e2
Use image.thumbnail
2023-03-22 16:03:05 -03:00
oobabooga
104212529f
Minor changes
2023-03-22 15:55:03 -03:00
wywywywy
61346b88ea
Add "seed" menu in the Parameters tab
2023-03-22 15:40:20 -03:00
Φφ
5389fce8e1
Extensions performance & memory optimisations
...
Reworked remove_surrounded_chars() to use regular expression ( https://regexr.com/7alb5 ) instead of repeated string concatenations for elevenlab_tts, silero_tts, sd_api_pictures. This should be both faster and more robust in handling asterisks.
Reduced the memory footprint of send_pictures and sd_api_pictures by scaling the images in the chat to 300 pixels max-side wise. (The user already has the original in case of the sent picture and there's an option to save the SD generation).
This should fix history growing annoyingly large with multiple pictures present
2023-03-22 11:51:00 +03:00
oobabooga
45b7e53565
Only catch proper Exceptions in the text generation function
2023-03-20 20:36:02 -03:00
oobabooga
6872ffd976
Update README.md
2023-03-20 16:53:14 -03:00
oobabooga
db4219a340
Update comments
2023-03-20 16:40:08 -03:00
oobabooga
7618f3fe8c
Add -gptq-preload for 4-bit offloading ( #460 )
...
This works in a 4GB card now:
```
python server.py --model llama-7b-hf --gptq-bits 4 --gptq-pre-layer 20
```
2023-03-20 16:30:56 -03:00
Vladimir Belitskiy
e96687b1d6
Do not send empty user input as part of the prompt.
...
However, if extensions modify the empty prompt to be non-empty,
it'l still work as before.
2023-03-20 14:27:39 -04:00
oobabooga
9a3bed50c3
Attempt at fixing 4-bit with CPU offload
2023-03-20 15:11:56 -03:00
oobabooga
536d0a4d93
Add an import
2023-03-20 14:00:40 -03:00
Vladimir Belitskiy
ca47e016b4
Do not display empty user messages in chat mode.
...
There doesn't seem to be much value to them - they just take up space while also making it seem like there's still some sort of pseudo-dialogue going on, instead of a monologue by the bot.
2023-03-20 12:55:57 -04:00
oobabooga
75a7a84ef2
Exception handling ( #454 )
...
* Update text_generation.py
* Update extensions.py
2023-03-20 13:36:52 -03:00
oobabooga
a90f507abe
Exit elevenlabs_tts if streaming is enabled
2023-03-20 11:49:42 -03:00
oobabooga
31ab2be8ef
Remove redundant requirements #309
2023-03-19 22:10:55 -03:00
oobabooga
164e05daad
Download .py files using download-model.py
2023-03-19 20:34:52 -03:00
oobabooga
dd4374edde
Update README
2023-03-19 20:15:15 -03:00
oobabooga
9378754cc7
Update README
2023-03-19 20:14:50 -03:00
oobabooga
7ddf6147ac
Update README.md
2023-03-19 19:25:52 -03:00
oobabooga
b552d2b58a
Remove unused imports
...
o
2023-03-19 19:24:41 -03:00
oobabooga
ddb62470e9
--no-cache and --gpu-memory in MiB for fine VRAM control
2023-03-19 19:21:41 -03:00
oobabooga
4bafe45a51
Merge pull request #309 from Brawlence/main
...
Extension: Stable Diffusion Api integration
2023-03-19 13:24:47 -03:00
oobabooga
eab8de0d4a
Merge branch 'main' into Brawlence-main
2023-03-19 13:09:59 -03:00
oobabooga
4d701a6eb9
Create a mirror for the preset menu
2023-03-19 12:51:47 -03:00
oobabooga
257edf5f56
Make the Default preset more reasonable
...
Credits: anonymous 4chan user who got it off
"some twitter post or something someone linked,
who even knows anymore"
2023-03-19 12:30:51 -03:00
oobabooga
a78b6508fc
Make custom LoRAs work by default #385
2023-03-19 12:11:35 -03:00
oobabooga
7073e96093
Add back RWKV dependency #98
2023-03-19 12:05:28 -03:00
oobabooga
217e1d9fdf
Merge branch 'main' of github.com:oobabooga/text-generation-webui
2023-03-19 10:37:23 -03:00
oobabooga
c79fc69e95
Fix the API example with streaming #417
2023-03-19 10:36:57 -03:00
Maya
acdbd6b708
Check if app should display extensions ui
2023-03-19 13:31:21 +00:00
Maya
81c9d130f2
Fix global
2023-03-19 13:25:49 +00:00
Maya
099d7a844b
Add setup method to extensions
2023-03-19 13:22:24 +00:00
Maya
bd27353a08
Fix duplicating server on ui reload
2023-03-19 12:51:27 +00:00
oobabooga
0cbe2dd7e9
Update README.md
2023-03-18 12:24:54 -03:00
oobabooga
36ac7be76d
Merge pull request #407 from ThisIsPIRI/gitignore
...
Add loras to .gitignore
2023-03-18 11:57:10 -03:00
oobabooga
d2a7fac8ea
Use pip instead of conda for pytorch
2023-03-18 11:56:04 -03:00
ThisIsPIRI
705f513c4c
Add loras to .gitignore
2023-03-18 23:33:24 +09:00
oobabooga
9ed3a03d4b
Don't use the official instructions
2023-03-18 11:25:08 -03:00
oobabooga
a0b1a30fd5
Specify torchvision/torchaudio versions
2023-03-18 11:23:56 -03:00
oobabooga
c753261338
Disable stop_at_newline by default
2023-03-18 10:55:57 -03:00
oobabooga
7c945cfe8e
Don't include PeftModel every time
2023-03-18 10:55:24 -03:00
