Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-11-30 03:18:57 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,852 Commits 3 Branches 52 Tags 67 MiB
3209440b7c
Commit Graph

46 Commits

Author SHA1 Message Date
oobabooga
3a6e194bc7
Change a warning message 2023-05-29 22:39:23 -03:00
oobabooga
acfd876f29 Some qol changes to "Perplexity evaluation" 2023-05-25 15:06:22 -03:00
oobabooga
63ce5f9c28 Add back a missing bos token 2023-05-24 13:54:36 -03:00
Alex "mcmonkey" Goodwin
3cd7c5bdd0
LoRA Trainer: train_only_after option to control which part of your input to train on (#2315) 2023-05-24 12:43:22 -03:00
oobabooga
e116d31180 Prevent unwanted log messages from modules 2023-05-21 22:42:34 -03:00
Alex "mcmonkey" Goodwin
50c70e28f0
Lora Trainer improvements, part 6 - slightly better raw text inputs (#2108) 2023-05-19 12:58:54 -03:00
Forkoz
d205ec9706
Fix Training fails when evaluation dataset is selected (#2099) 
Fixes https://github.com/oobabooga/text-generation-webui/issues/2078 from Googulator
 2023-05-16 13:40:19 -03:00
oobabooga
56f6b7052a Sort dropdowns numerically 2023-05-05 23:14:56 -03:00
oobabooga
95d04d6a8d Better warning messages 2023-05-03 21:43:17 -03:00
practicaldreamer
e3968f7dd0
Fix Training Pad Token (#1678) 
Currently padding with 0 the character vs 0 the token id (<unk> in the case of llama)
 2023-05-02 23:16:08 -03:00
Alex "mcmonkey" Goodwin
312cb7dda6
LoRA trainer improvements part 5 (#1546) 
* full dynamic model type support on modern peft

* remove shuffle option
 2023-04-25 21:27:30 -03:00
Alex "mcmonkey" Goodwin
459e725af9
Lora trainer docs (#1493) 2023-04-23 12:54:41 -03:00
oobabooga
d46b9b7c50 Fix evaluate comment saving 2023-04-21 12:34:08 -03:00
oobabooga
c4f4f41389
Add an "Evaluate" tab to calculate the perplexities of models (#1322) 2023-04-21 00:20:33 -03:00
Alex "mcmonkey" Goodwin
ee30625cd1
4-Bit LoRA training + several new training options and fixes 2023-04-19 19:39:03 -03:00
Tynan Burke
6a810b16b2
typo in training.py (#1329) 2023-04-17 21:40:46 -03:00
oobabooga
b705b4210c Minor changes to training.py 2023-04-16 03:08:37 -03:00
oobabooga
5c513a5f5c Make training.py more readable 2023-04-16 02:46:27 -03:00
Alex "mcmonkey" Goodwin
a3eec62b50
Lora trainer improvements part 3 (#1098) 
* add support for other model types

dependent on future-peft-changes but with fallback to function now

* use encoding=utf8 for training format

* make shuffling optional

and describe dropout a bit more

* add eval_steps to control evaluation

* make callbacks not depend on globals

* make save steps controllable

* placeholder of initial loading-existing-model support

and var name cleanup

* save/load parameters

* last bit of cleanup

* remove `gptq_bits` ref as main branch removed that setting

* add higher_rank_limit option

2048 is basically unreachable due to VRAM, but i trained at 1536 with batch size = 1 on a 7B model.
Note that it's in the do_train input just to save as a parameter

* fix math on save_steps
 2023-04-16 02:35:13 -03:00
oobabooga
abef355ed0 Remove deprecated flag 2023-04-15 01:21:19 -03:00
Lukas
5ad92c940e
lora training fixes: (#970) 
Fix wrong input format being picked
Fix crash when an entry in the dataset has an attribute of value None
 2023-04-12 11:38:01 -03:00
IggoOnCode
09d8119e3c
Add CPU LoRA training (#938) 
(It's very slow)
 2023-04-10 17:29:00 -03:00
oobabooga
768354239b Change training file encoding 2023-04-07 11:15:52 -03:00
oobabooga
ea6e77df72
Make the code more like PEP8 for readability (#862) 2023-04-07 00:15:45 -03:00
Alex "mcmonkey" Goodwin
0c7ef26981
Lora trainer improvements (#763) 2023-04-06 02:04:11 -03:00
oobabooga
2a267011dc Use Path.stem for simplicity 2023-04-03 00:56:14 -03:00
oobabooga
58349f44a0
Handle training exception for unsupported models 2023-03-29 11:55:34 -03:00
oobabooga
a6d0373063
Fix training dataset loading #636 2023-03-29 11:48:17 -03:00
Alex "mcmonkey" Goodwin
e817fac542 better defaults 2023-03-27 22:29:23 -07:00
Alex "mcmonkey" Goodwin
2e08af4edf implement initial Raw Text File Input 
also bump default Rank & Alpha for values that will make sense in testing if you don't know what you're doing and leave the defaults.
 2023-03-27 22:15:32 -07:00
Alex "mcmonkey" Goodwin
b749952fe3 change number minimums to 0 
gradio calculates 'step' relative to the minimum, so at '1' the step values were all offset awkwardly. 0 isn't valid, but, uh, just don't slam the slider to the left.
 2023-03-27 21:22:43 -07:00
Alex "mcmonkey" Goodwin
ec6224f556 use new shared.args.lora_dir 2023-03-27 20:04:16 -07:00
Alex "mcmonkey" Goodwin
8a97f6ba29 corrections per the PR comments 2023-03-27 18:39:06 -07:00
Alex "mcmonkey" Goodwin
7fab7ea1b6 couple missed camelCases 2023-03-27 18:19:06 -07:00
Alex "mcmonkey" Goodwin
6368dad7db Fix camelCase to snake_case to match repo format standard 2023-03-27 18:17:42 -07:00
oobabooga
2f0571bfa4 Small style changes 2023-03-27 21:24:39 -03:00
Alex "mcmonkey" Goodwin
9ced75746d add total time estimate 2023-03-27 10:57:27 -07:00
Alex "mcmonkey" Goodwin
16ea4fc36d interrupt button 2023-03-27 10:43:01 -07:00
Alex "mcmonkey" Goodwin
8fc723fc95 initial progress tracker in UI 2023-03-27 10:25:08 -07:00
Alex "mcmonkey" Goodwin
c07bcd0850 add some outputs to indicate progress updates (sorta) 
Actual progressbar still needed. Also minor formatting fixes.
 2023-03-27 09:41:06 -07:00
Alex "mcmonkey" Goodwin
d911c22af9 use shared rows to make the LoRA Trainer interface a bit more compact / clean 2023-03-27 08:31:49 -07:00
Alex "mcmonkey" Goodwin
f1ba2196b1 make 'model' variables less ambiguous 2023-03-25 12:57:36 -07:00
Alex "mcmonkey" Goodwin
8da237223e document options better 2023-03-25 12:48:35 -07:00
Alex "mcmonkey" Goodwin
5c49a0dcd0 fix error from prepare call running twice in a row 2023-03-25 12:37:32 -07:00
Alex "mcmonkey" Goodwin
7bf601107c automatically strip empty data entries (for better alpaca dataset compat) 2023-03-25 12:28:46 -07:00
Alex "mcmonkey" Goodwin
566898a79a initial lora training tab 2023-03-25 12:08:26 -07:00