This website requires JavaScript.
Explore
Help
Register
Sign In
Mirrors
/
text-generation-webui
Watch
1
Star
0
Fork
0
You've already forked text-generation-webui
mirror of
https://github.com/oobabooga/text-generation-webui.git
synced
2024-11-27 10:09:14 +01:00
Code
Issues
Packages
Projects
Releases
Wiki
Activity
768124c4b0
text-generation-webui
/
training
/
datasets
/
put-trainer-datasets-here.txt
2 lines
78 B
Plaintext
Raw
Normal View
History
Unescape
Escape
Add ability to load all text files from a subdirectory for training (#1997) * Update utils.py returns individual txt files and subdirectories to getdatasets to allow for training from a directory of text files * Update training.py minor tweak to training on raw datasets to detect if a directory is selected, and if so, to load in all the txt files in that directory for training * Update put-trainer-datasets-here.txt document * Minor change * Use pathlib, sort by natural keys * Space --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2023-07-12 16:44:30 +02:00
to load multiple raw text files create a subdirectory and put them all there
Reference in New Issue
Copy Permalink