From 6ec997f195e218b868819302a132189dc12f9332 Mon Sep 17 00:00:00 2001 From: oobabooga <112222186+oobabooga@users.noreply.github.com> Date: Tue, 7 Nov 2023 12:36:52 -0300 Subject: [PATCH 1/4] Update 12 - OpenAI API.md --- docs/12 - OpenAI API.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/12 - OpenAI API.md b/docs/12 - OpenAI API.md index 09afc2a9..12a6c46a 100644 --- a/docs/12 - OpenAI API.md +++ b/docs/12 - OpenAI API.md @@ -180,7 +180,7 @@ while True: print() ``` -### Python completions example with streaming +#### Python completions example with streaming Start the script with `python -u` to see the output in real time. From 40e73aafce16f7b8fd49909c36628a1c5b08ce6b Mon Sep 17 00:00:00 2001 From: oobabooga <112222186+oobabooga@users.noreply.github.com> Date: Tue, 7 Nov 2023 12:38:39 -0300 Subject: [PATCH 2/4] Update 12 - OpenAI API.md --- docs/12 - OpenAI API.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/docs/12 - OpenAI API.md b/docs/12 - OpenAI API.md index 12a6c46a..0d90b942 100644 --- a/docs/12 - OpenAI API.md +++ b/docs/12 - OpenAI API.md @@ -215,8 +215,7 @@ for event in client.events(): print() ``` -### Client Application Setup - +### Third-party application setup You can usually force an application that uses the OpenAI API to connect to the local API by using the following environment variables: @@ -228,18 +227,18 @@ or ```shell OPENAI_API_KEY=sk-111111111111111111111111111111111111111111111111 -OPENAI_API_BASE=http://127.0.0.1:500/v1 +OPENAI_API_BASE=http://127.0.0.1:5000/v1 ``` -With the [official python openai client](https://github.com/openai/openai-python), set the `OPENAI_API_BASE` environment variables: +With the [official python openai client](https://github.com/openai/openai-python), the address can be set like this: ```shell -# Sample .env file: -OPENAI_API_KEY=sk-111111111111111111111111111111111111111111111111 -OPENAI_API_BASE=http://0.0.0.0:5001/v1 -``` +import openai -If needed, replace 127.0.0.1 with the IP/port of your server. +openai.api_key = "..." +openai.api_base = "http://127.0.0.1:5000/v1" +openai.api_version = "2023-05-15" +``` If using .env files to save the `OPENAI_API_BASE` and `OPENAI_API_KEY` variables, make sure the .env file is loaded before the openai module is imported: From ddca6948b2a7077e8cab82f8bb9a721c767c41ba Mon Sep 17 00:00:00 2001 From: oobabooga <112222186+oobabooga@users.noreply.github.com> Date: Tue, 7 Nov 2023 12:39:59 -0300 Subject: [PATCH 3/4] Update 12 - OpenAI API.md --- docs/12 - OpenAI API.md | 30 +----------------------------- 1 file changed, 1 insertion(+), 29 deletions(-) diff --git a/docs/12 - OpenAI API.md b/docs/12 - OpenAI API.md index 0d90b942..5cdc4f6a 100644 --- a/docs/12 - OpenAI API.md +++ b/docs/12 - OpenAI API.md @@ -44,7 +44,7 @@ openai-debug: 1 ### Examples -For the documentation with all the parameters, consult `http://127.0.0.1:5000/docs` or the [typing.py](https://github.com/oobabooga/text-generation-webui/blob/main/extensions/openai/typing.py) file. +For the documentation with all the parameters and their types, consult `http://127.0.0.1:5000/docs` or the [typing.py](https://github.com/oobabooga/text-generation-webui/blob/main/extensions/openai/typing.py) file. The official examples in the [OpenAI documentation](https://platform.openai.com/docs/api-reference) should also work, and the same parameters apply (although the API here has more optional parameters). @@ -282,33 +282,6 @@ In short, the all-MiniLM-L6-v2 model is 5x faster, 5x smaller ram, 2x smaller st Warning: You cannot mix embeddings from different models even if they have the same dimensions. They are not comparable. -### API Documentation & Examples - -The OpenAI API is well documented, you can view the documentation here: https://platform.openai.com/docs/api-reference - -Examples of how to use the Completions API in Python can be found here: https://platform.openai.com/examples -Not all of them will work with all models unfortunately, See the notes on Models for how to get the best results. - -Here is a simple python example. - -```python -import os -os.environ['OPENAI_API_KEY']="sk-111111111111111111111111111111111111111111111111" -os.environ['OPENAI_API_BASE']="http://0.0.0.0:5001/v1" -import openai - -response = openai.ChatCompletion.create( - model="x", - messages = [{ 'role': 'system', 'content': "Answer in a consistent style." }, - {'role': 'user', 'content': "Teach me about patience."}, - {'role': 'assistant', 'content': "The river that carves the deepest valley flows from a modest spring; the grandest symphony originates from a single note; the most intricate tapestry begins with a solitary thread."}, - {'role': 'user', 'content': "Teach me about the ocean."}, - ] -) -text = response['choices'][0]['message']['content'] -print(text) -``` - ### Compatibility & not so compatibility | API endpoint | tested with | notes | @@ -333,7 +306,6 @@ print(text) | /v1/fine-tunes\* | openai.FineTune.\* | not yet supported | | /v1/search | openai.search, engines.search | not yet supported | - #### Applications Almost everything needs the `OPENAI_API_KEY` and `OPENAI_API_BASE` environment variable set, but there are some exceptions. From cc04abda4949c745cf93106fcd0bf96c60595029 Mon Sep 17 00:00:00 2001 From: oobabooga <112222186+oobabooga@users.noreply.github.com> Date: Tue, 7 Nov 2023 12:40:52 -0300 Subject: [PATCH 4/4] Update 12 - OpenAI API.md --- docs/12 - OpenAI API.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/12 - OpenAI API.md b/docs/12 - OpenAI API.md index 5cdc4f6a..a4365ed3 100644 --- a/docs/12 - OpenAI API.md +++ b/docs/12 - OpenAI API.md @@ -284,6 +284,8 @@ Warning: You cannot mix embeddings from different models even if they have the s ### Compatibility & not so compatibility +Note: the table below may be obsolete. + | API endpoint | tested with | notes | | ------------------------- | ---------------------------------- | --------------------------------------------------------------------------- | | /v1/chat/completions | openai.ChatCompletion.create() | Use it with instruction following models | @@ -310,6 +312,8 @@ Warning: You cannot mix embeddings from different models even if they have the s Almost everything needs the `OPENAI_API_KEY` and `OPENAI_API_BASE` environment variable set, but there are some exceptions. +Note: the table below may be obsolete. + | Compatibility | Application/Library | Website | Notes | | ------------- | ---------------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ✅❌ | openai-python (v0.25+) | https://github.com/openai/openai-python | only the endpoints from above are working. OPENAI_API_BASE=http://127.0.0.1:5001/v1 |