Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-11-24 17:06:53 +01:00
Code Issues Packages Projects Releases Wiki Activity

Simplified ExLlama cloning instructions and failure message (#3972)

Browse Source
This commit is contained in:
James Braza 2023-09-17 15:26:05 -07:00 committed by GitHub
parent 45335fa8f4
commit fee38e0601
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 6 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -101,9 +101,7 @@ pip install -r requirements_nocuda.txt
``` ```
cd text-generation-webui cd text-generation-webui
mkdir repositories git clone https://github.com/turboderp/exllama repositories/exllama
cd repositories
git clone https://github.com/turboderp/exllama
``` ```
#### bitsandbytes on older NVIDIA GPUs #### bitsandbytes on older NVIDIA GPUs

7
modules/exllama.py
View File

@ -14,7 +14,7 @@ try:
from exllama.model import ExLlama, ExLlamaCache, ExLlamaConfig from exllama.model import ExLlama, ExLlamaCache, ExLlamaConfig
from exllama.tokenizer import ExLlamaTokenizer from exllama.tokenizer import ExLlamaTokenizer
except: except:
logger.warning('Exllama module failed to load. Will attempt to load from repositories.') logger.warning('exllama module failed to import. Will attempt to import from repositories/.')
try: try:
from modules.relative_imports import RelativeImport from modules.relative_imports import RelativeImport
@ -23,7 +23,10 @@ except:
from model import ExLlama, ExLlamaCache, ExLlamaConfig from model import ExLlama, ExLlamaCache, ExLlamaConfig
from tokenizer import ExLlamaTokenizer from tokenizer import ExLlamaTokenizer
except: except:
logger.error("Could not find repositories/exllama/. Make sure that exllama is cloned inside repositories/ and is up to date.") logger.error(
"Could not find repositories/exllama. Please ensure that exllama"
" (https://github.com/turboderp/exllama) is cloned inside repositories/ and is up to date."
)
raise raise