Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-12-23 21:18:00 +01:00
Code Issues Packages Projects Releases Wiki Activity

Update RWKV-model.md

Browse Source
This commit is contained in:
oobabooga 2023-05-23 02:10:08 -03:00 committed by GitHub
parent cd3618d7fb
commit b0845ae4e8
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/RWKV-model.md
View File

@ -46,7 +46,7 @@ No additional steps are required. Just launch it as you would with any other mod
python server.py --listen --no-stream --model RWKV-4-Pile-169M-20220807-8023.pth
```
### Setting a custom strategy
#### Setting a custom strategy
It is possible to have very fine control over the offloading and precision for the model with the `--rwkv-strategy` flag. Possible values include:
@ -59,6 +59,6 @@ It is possible to have very fine control over the offloading and precision for t
See the README for the PyPl package for more details: https://pypi.org/project/rwkv/
### Compiling the CUDA kernel
#### Compiling the CUDA kernel
You can compile the CUDA kernel for the model with `--rwkv-cuda-on`. This should improve the performance a lot but I haven't been able to get it to work yet.
You can compile the CUDA kernel for the model with `--rwkv-cuda-on`. This should improve the performance a lot but I haven't been able to get it to work yet.