Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-10-29 21:50:16 +01:00
Code Issues Packages Projects Releases Wiki Activity

Update README

Browse Source
This commit is contained in:
oobabooga 2023-01-09 18:18:35 -03:00
parent fbbad7cf8a
commit 9aa51ca227
1 changed files with 1 additions and 45 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
README.md
View File

@ -98,51 +98,7 @@ Inference settings presets can be created under `presets/` as text files. These
## System requirements
These are the VRAM (in GiB) and RAM (in MiB) requirements to run some model examples.
#### GPU mode (default)
| model | VRAM (GPU) | RAM |
|:-----------------------|-------------:|--------:|
| OPT-350M-Erebus | 0.62 | 1939.3 |
| arxiv_ai_gpt2 | 1.48 | 6350.68 |
| blenderbot-1B-distill | 2.38 | 2705.9 |
| opt-1.3b | 2.45 | 2868.12 |
| gpt-neo-1.3b | 2.54 | 4047.04 |
| gpt4chan_model_float16 | 11.38 | 1909.79 |
| gpt-j-6b-float16 | 11.38 | 2847.75 |
| gpt-j-6B | 11.38 | 3959.55 |
| galactica-6.7b | 12.4 | 1933.19 |
| opt-6.7b | 12.4 | 1944.21 |
| bloomz-7b1-p3 | 13.17 | 1845.58 |
#### GPU mode with 8-bit precision
Allows you to load models that would not normally fit into your GPU. Enabled by default for 13b and 20b models in this webui.
| model | VRAM (GPU) | RAM |
|:---------------|-------------:|--------:|
| OPT-13B-Erebus | 12.23 | 749.08 |
| opt-13b | 12.23 | 1258.95 |
| gpt-neox-20b | 19.91 | 2104.04 |
#### CPU mode
A lot slower, but does not require a GPU.
| model | RAM |
|:-----------------------|---------:|
| OPT-350M-Erebus | 2622.17 |
| arxiv_ai_gpt2 | 3764.81 |
| gpt-neo-1.3b | 5937.81 |
| opt-1.3b | 7346.08 |
| blenderbot-1B-distill | 7565.36 |
| bloomz-7b1-p3 | 23613.9 |
| gpt-j-6B | 23975.5 |
| gpt4chan_model | 23999.5 |
| gpt-j-6b-float16 | 24999.1 |
| galactica-6.7b | 26248 |
| opt-6.7b | 27334.2 |
Check the [wiki](https://github.com/oobabooga/text-generation-webui/wiki/System-requirements) for some examples of VRAM and RAM usage in both GPU and CPU mode.
## Contributing