mirror of
https://github.com/oobabooga/text-generation-webui.git
synced 2024-11-25 09:19:23 +01:00
added windows/docker docs (#1027)
This commit is contained in:
loeken
GitHub
parent
b0c762ceba
commit
89e22d4d6a
@ -120,13 +120,13 @@ As an alternative to the recommended WSL method, you can install the web UI nati
|
||||
### Alternative: Docker
|
||||
|
||||
```
|
||||
cp .env.example .env
|
||||
cp docker/{Dockerfile,docker-compose.yml,.dockerignore} .
|
||||
cp docker/.env.example .env
|
||||
# Edit .env and set TORCH_CUDA_ARCH_LIST based on your GPU model
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Make sure to edit `.env.example` and set the appropriate CUDA version for your GPU, which can be found on [developer.nvidia.com](https://developer.nvidia.com/cuda-gpus).
|
||||
|
||||
You need to have docker compose v2.17 or higher installed in your system. For installation instructions, see [Docker compose installation](https://github.com/oobabooga/text-generation-webui/wiki/Docker-compose-installation).
|
||||
You need to have docker compose v2.17 or higher installed in your system. To see how to install docker compose itself, see the guide in https://github.com/oobabooga/text-generation-webui/docker/.
|
||||
|
||||
Contributed by [@loeken](https://github.com/loeken) in [#633](https://github.com/oobabooga/text-generation-webui/pull/633)
|
||||
|
||||
|
||||
170
docker/README.md
Normal file
170
docker/README.md
Normal file
@ -0,0 +1,170 @@
|
||||
- [Ubuntu 22.04](#ubuntu-2204)
|
||||
- [0. youtube video](#0-youtube-video)
|
||||
- [1. update the drivers](#1-update-the-drivers)
|
||||
- [2. reboot](#2-reboot)
|
||||
- [3. install docker](#3-install-docker)
|
||||
- [4. docker \& container toolkit](#4-docker--container-toolkit)
|
||||
- [5. clone the repo](#5-clone-the-repo)
|
||||
- [6. prepare models](#6-prepare-models)
|
||||
- [7. prepare .env file](#7-prepare-env-file)
|
||||
- [8. startup docker container](#8-startup-docker-container)
|
||||
- [Manjaro](#manjaro)
|
||||
- [update the drivers](#update-the-drivers)
|
||||
- [reboot](#reboot)
|
||||
- [docker \& container toolkit](#docker--container-toolkit)
|
||||
- [continue with ubuntu task](#continue-with-ubuntu-task)
|
||||
- [Windows](#windows)
|
||||
- [0. youtube video](#0-youtube-video-1)
|
||||
- [1. choco package manager](#1-choco-package-manager)
|
||||
- [2. install drivers/dependencies](#2-install-driversdependencies)
|
||||
- [3. install wsl](#3-install-wsl)
|
||||
- [4. reboot](#4-reboot)
|
||||
- [5. git clone \&\& startup](#5-git-clone--startup)
|
||||
- [6. prepare models](#6-prepare-models-1)
|
||||
- [7. startup](#7-startup)
|
||||
- [notes](#notes)
|
||||
|
||||
# Ubuntu 22.04
|
||||
|
||||
## 0. youtube video
|
||||
A video walking you through the setup can be found here:
|
||||
|
||||
[](https://www.youtube.com/watch?v=ELkKWYh8qOk)
|
||||
|
||||
|
||||
## 1. update the drivers
|
||||
in the the “software updater” update drivers to the last version of the prop driver.
|
||||
|
||||
## 2. reboot
|
||||
to switch using to new driver
|
||||
|
||||
## 3. install docker
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt-get install curl
|
||||
sudo mkdir -m 0755 -p /etc/apt/keyrings
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
||||
echo \
|
||||
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
|
||||
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
|
||||
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||
sudo apt update
|
||||
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-compose -y
|
||||
sudo usermod -aG docker $USER
|
||||
newgrp docker
|
||||
```
|
||||
|
||||
## 4. docker & container toolkit
|
||||
```bash
|
||||
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
|
||||
echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/ubuntu22.04/amd64 /" | \
|
||||
sudo tee /etc/apt/sources.list.d/nvidia.list > /dev/null
|
||||
sudo apt update
|
||||
sudo apt install nvidia-docker2 nvidia-container-runtime -y
|
||||
sudo systemctl restart docker
|
||||
```
|
||||
|
||||
## 5. clone the repo
|
||||
```
|
||||
git clone https://github.com/oobabooga/text-generation-webui
|
||||
cd text-generation-webui
|
||||
```
|
||||
|
||||
## 6. prepare models
|
||||
download and place the models inside the models folder. tested with:
|
||||
|
||||
4bit
|
||||
https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617
|
||||
https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105
|
||||
|
||||
8bit:
|
||||
https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789
|
||||
|
||||
## 7. prepare .env file
|
||||
edit .env values to your needs.
|
||||
```bash
|
||||
cp .env.example .env
|
||||
nano .env
|
||||
```
|
||||
|
||||
## 8. startup docker container
|
||||
```bash
|
||||
docker-compose up --build
|
||||
```
|
||||
|
||||
# Manjaro
|
||||
manjaro/arch is similar to ubuntu just the dependency installation is more convenient
|
||||
|
||||
## update the drivers
|
||||
```bash
|
||||
sudo mhwd -a pci nonfree 0300
|
||||
```
|
||||
## reboot
|
||||
```bash
|
||||
reboot
|
||||
```
|
||||
## docker & container toolkit
|
||||
```bash
|
||||
yay -S docker docker-compose buildkit gcc nvidia-docker
|
||||
sudo usermod -aG docker $USER
|
||||
newgrp docker
|
||||
sudo systemctl restart docker # required by nvidia-container-runtime
|
||||
```
|
||||
|
||||
## continue with ubuntu task
|
||||
continue at [5. clone the repo](#5-clone-the-repo)
|
||||
|
||||
# Windows
|
||||
## 0. youtube video
|
||||
A video walking you through the setup can be found here:
|
||||
[](https://www.youtube.com/watch?v=ejH4w5b5kFQ)
|
||||
|
||||
## 1. choco package manager
|
||||
install package manager (https://chocolatey.org/ )
|
||||
```
|
||||
Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
|
||||
```
|
||||
|
||||
## 2. install drivers/dependencies
|
||||
```
|
||||
choco install nvidia-display-driver cuda git docker-desktop
|
||||
```
|
||||
|
||||
## 3. install wsl
|
||||
wsl --install
|
||||
|
||||
## 4. reboot
|
||||
after reboot enter username/password in wsl
|
||||
|
||||
## 5. git clone && startup
|
||||
clone the repo and edit .env values to your needs.
|
||||
```
|
||||
cd Desktop
|
||||
git clone https://github.com/oobabooga/text-generation-webui
|
||||
cd text-generation-webui
|
||||
COPY .env.example .env
|
||||
notepad .env
|
||||
```
|
||||
|
||||
## 6. prepare models
|
||||
download and place the models inside the models folder. tested with:
|
||||
|
||||
4bit https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617 https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105
|
||||
|
||||
8bit: https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789
|
||||
|
||||
## 7. startup
|
||||
```
|
||||
docker-compose up
|
||||
```
|
||||
|
||||
# notes
|
||||
|
||||
on older ubuntus you can manually install the docker compose plugin like this:
|
||||
```
|
||||
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
|
||||
mkdir -p $DOCKER_CONFIG/cli-plugins
|
||||
curl -SL https://github.com/docker/compose/releases/download/v2.17.2/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
|
||||
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
|
||||
export PATH="$HOME/.docker/cli-plugins:$PATH"
|
||||
```
|
||||
Loading…
Reference in New Issue
Block a user