Mirrors/text-generation-webui
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-11-24 17:06:53 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'refs/remotes/origin/dev' into dev

Browse Source
This commit is contained in:
oobabooga 2023-11-07 07:46:08 -08:00
commit b0b999dd68
1 changed files with 14 additions and 39 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
docs/12 - OpenAI API.md
View File

@ -45,7 +45,7 @@ openai-debug: 1
### Examples
For the documentation with all the parameters, consult `http://127.0.0.1:5000/docs` or the [typing.py](https://github.com/oobabooga/text-generation-webui/blob/main/extensions/openai/typing.py) file.
For the documentation with all the parameters and their types, consult `http://127.0.0.1:5000/docs` or the [typing.py](https://github.com/oobabooga/text-generation-webui/blob/main/extensions/openai/typing.py) file.
The official examples in the [OpenAI documentation](https://platform.openai.com/docs/api-reference) should also work, and the same parameters apply (although the API here has more optional parameters).
@ -181,7 +181,7 @@ while True:
print()
```
### Python completions example with streaming
#### Python completions example with streaming
Start the script with `python -u` to see the output in real time.
@ -216,8 +216,7 @@ for event in client.events():
print()
```
### Client Application Setup
### Third-party application setup
You can usually force an application that uses the OpenAI API to connect to the local API by using the following environment variables:
@ -229,18 +228,18 @@ or
```shell
OPENAI_API_KEY=sk-111111111111111111111111111111111111111111111111
OPENAI_API_BASE=http://127.0.0.1:500/v1
OPENAI_API_BASE=http://127.0.0.1:5000/v1
```
With the [official python openai client](https://github.com/openai/openai-python), set the `OPENAI_API_BASE` environment variables:
With the [official python openai client](https://github.com/openai/openai-python), the address can be set like this:
```shell
# Sample .env file:
OPENAI_API_KEY=sk-111111111111111111111111111111111111111111111111
OPENAI_API_BASE=http://0.0.0.0:5001/v1
```
import openai
If needed, replace 127.0.0.1 with the IP/port of your server.
openai.api_key = "..."
openai.api_base = "http://127.0.0.1:5000/v1"
openai.api_version = "2023-05-15"
```
If using .env files to save the `OPENAI_API_BASE` and `OPENAI_API_KEY` variables, make sure the .env file is loaded before the openai module is imported:
@ -284,35 +283,10 @@ In short, the all-MiniLM-L6-v2 model is 5x faster, 5x smaller ram, 2x smaller st
Warning: You cannot mix embeddings from different models even if they have the same dimensions. They are not comparable.
### API Documentation & Examples
The OpenAI API is well documented, you can view the documentation here: https://platform.openai.com/docs/api-reference
Examples of how to use the Completions API in Python can be found here: https://platform.openai.com/examples
Not all of them will work with all models unfortunately, See the notes on Models for how to get the best results.
Here is a simple python example.
```python
import os
os.environ['OPENAI_API_KEY']="sk-111111111111111111111111111111111111111111111111"
os.environ['OPENAI_API_BASE']="http://0.0.0.0:5001/v1"
import openai
response = openai.ChatCompletion.create(
model="x",
messages = [{ 'role': 'system', 'content': "Answer in a consistent style." },
{'role': 'user', 'content': "Teach me about patience."},
{'role': 'assistant', 'content': "The river that carves the deepest valley flows from a modest spring; the grandest symphony originates from a single note; the most intricate tapestry begins with a solitary thread."},
{'role': 'user', 'content': "Teach me about the ocean."},
]
)
text = response['choices'][0]['message']['content']
print(text)
```
### Compatibility & not so compatibility
Note: the table below may be obsolete.
| API endpoint | tested with | notes |
| ------------------------- | ---------------------------------- | --------------------------------------------------------------------------- |
| /v1/chat/completions | openai.ChatCompletion.create() | Use it with instruction following models |
@ -335,11 +309,12 @@ print(text)
| /v1/fine-tunes\* | openai.FineTune.\* | not yet supported |
| /v1/search | openai.search, engines.search | not yet supported |
#### Applications
Almost everything needs the `OPENAI_API_KEY` and `OPENAI_API_BASE` environment variable set, but there are some exceptions.
Note: the table below may be obsolete.
| Compatibility | Application/Library | Website | Notes |
| ------------- | ---------------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| ✅❌ | openai-python (v0.25+) | https://github.com/openai/openai-python | only the endpoints from above are working. OPENAI_API_BASE=http://127.0.0.1:5001/v1 |