From 55dc9845cb895d4cc2e3e145be1ee9f8db648e4c Mon Sep 17 00:00:00 2001 From: oobabooga <112222186+oobabooga@users.noreply.github.com> Date: Tue, 7 Nov 2023 12:51:41 -0300 Subject: [PATCH 1/3] Update 12 - OpenAI API.md --- docs/12 - OpenAI API.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/12 - OpenAI API.md b/docs/12 - OpenAI API.md index 17c83a3d..90e171fc 100644 --- a/docs/12 - OpenAI API.md +++ b/docs/12 - OpenAI API.md @@ -174,11 +174,15 @@ while True: stream_response = requests.post(url, headers=headers, json=data, verify=False, stream=True) client = sseclient.SSEClient(stream_response) + assistant_message = '' for event in client.events(): payload = json.loads(event.data) - print(payload['choices'][0]['message']['content'], end='') + chunk = payload['choices'][0]['message']['content'] + assistant_message += chunk + print(chunk, end='') print() + history.append({"role": "assistant", "content": assistant_message}) ``` #### Python completions example with streaming From 0c440877de5f4903cf6a568c905a24fd8f406d08 Mon Sep 17 00:00:00 2001 From: oobabooga <112222186+oobabooga@users.noreply.github.com> Date: Tue, 7 Nov 2023 12:59:40 -0300 Subject: [PATCH 2/3] Update 12 - OpenAI API.md --- docs/12 - OpenAI API.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/12 - OpenAI API.md b/docs/12 - OpenAI API.md index 90e171fc..c0261785 100644 --- a/docs/12 - OpenAI API.md +++ b/docs/12 - OpenAI API.md @@ -237,7 +237,7 @@ OPENAI_API_BASE=http://127.0.0.1:5000/v1 With the [official python openai client](https://github.com/openai/openai-python), the address can be set like this: -```shell +```python import openai openai.api_key = "..." From d59f1ad89a0d68c5a0f8883a5dd622b65cd41555 Mon Sep 17 00:00:00 2001 From: oobabooga <112222186+oobabooga@users.noreply.github.com> Date: Tue, 7 Nov 2023 13:05:06 -0300 Subject: [PATCH 3/3] Update README.md --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index f8a691a0..95b9a12b 100644 --- a/README.md +++ b/README.md @@ -20,9 +20,8 @@ Its goal is to become the [AUTOMATIC1111/stable-diffusion-webui](https://github. * [Multimodal pipelines, including LLaVA and MiniGPT-4](https://github.com/oobabooga/text-generation-webui/tree/main/extensions/multimodal) * [Extensions framework](https://github.com/oobabooga/text-generation-webui/wiki/07-%E2%80%90-Extensions) * [Custom chat characters](https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-Tab#character) -* Very efficient text streaming * Markdown output with LaTeX rendering, to use for instance with [GALACTICA](https://github.com/paperswithcode/galai) -* OpenAI-compatible API server +* OpenAI-compatible API server with Chat and Completions endpoints -- see the [examples](https://github.com/oobabooga/text-generation-webui/wiki/12-%E2%80%90-OpenAI-API#examples) ## Documentation