248 Commits

Author	SHA1	Message	Date
oobabooga	9992f7d8c0	Improve several log messages	2023-12-19 20:54:32 -08:00
oobabooga	de138b8ba6	Add llama-cpp-python wheels with tensor cores support (#5003)	2023-12-19 17:30:53 -03:00
oobabooga	0a299d5959	Bump llama-cpp-python to 0.2.24 (#5001)	2023-12-19 15:22:21 -03:00
oobabooga	a23a004434	Update the example template	2023-12-18 17:47:35 -08:00
Water	674be9a09a	Add HQQ quant loader (#4888) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2023-12-18 21:23:16 -03:00
oobabooga	f1f2c4c3f4	Add --num_experts_per_token parameter (ExLlamav2) (#4955)	2023-12-17 12:08:33 -03:00
oobabooga	3bbf6c601d	AutoGPTQ: Add --disable_exllamav2 flag (Mixtral CPU offloading needs this)	2023-12-15 06:46:13 -08:00
oobabooga	1c531a3713	Minor cleanup	2023-12-12 13:25:21 -08:00
oobabooga	39d2fe1ed9	Jinja templates for Instruct and Chat (#4874)	2023-12-12 17:23:14 -03:00
oobabooga	8c8825b777	Add QuIP# to README	2023-12-08 08:40:42 -08:00
oobabooga	2c5a1e67f9	Parameters: change max_new_tokens & repetition_penalty_range defaults (#4842)	2023-12-07 20:04:52 -03:00
oobabooga	98361af4d5	Add QuIP# support (#4803) ... It has to be installed manually for now.	2023-12-06 00:01:01 -03:00
oobabooga	131a5212ce	UI: update context upper limit to 200000	2023-12-04 15:48:34 -08:00
oobabooga	be88b072e9	Update --loader flag description	2023-12-04 15:41:25 -08:00
Lounger	7c0a17962d	Gallery improvements (#4789)	2023-12-03 22:45:50 -03:00
oobabooga	2769a1fa25	Hide deprecated args from Session tab	2023-11-21 15:15:16 -08:00
oobabooga	ef6feedeb2	Add --nowebui flag for pure API mode (#4651)	2023-11-18 23:38:39 -03:00
oobabooga	8f4f4daf8b	Add --admin-key flag for API (#4649)	2023-11-18 22:33:27 -03:00
oobabooga	e0ca49ed9c	Bump llama-cpp-python to 0.2.18 (2nd attempt) (#4637) ... * Update requirements*.txt * Add back seed	2023-11-18 00:31:27 -03:00
oobabooga	9d6f79db74	Revert "Bump llama-cpp-python to 0.2.18 (#4611)" ... This reverts commit 923c8e25fb.	2023-11-17 05:14:25 -08:00
oobabooga	13dc3b61da	Update README	2023-11-16 19:57:55 -08:00
oobabooga	8b66d83aa9	Set use_fast=True by default, create --no_use_fast flag ... This increases tokens/second for HF loaders.	2023-11-16 19:55:28 -08:00
oobabooga	923c8e25fb	Bump llama-cpp-python to 0.2.18 (#4611)	2023-11-16 22:55:14 -03:00
oobabooga	4aabff3728	Remove old API, launch OpenAI API with --api	2023-11-10 06:39:08 -08:00
oobabooga	6e2e0317af	Separate context and system message in instruction formats (#4499)	2023-11-07 20:02:58 -03:00
oobabooga	af3d25a503	Disable logits_all in llamacpp_HF (makes processing 3x faster)	2023-11-07 14:35:48 -08:00
oobabooga	ec17a5d2b7	Make OpenAI API the default API (#4430)	2023-11-06 02:38:29 -03:00
feng lui	4766a57352	transformers: add use_flash_attention_2 option (#4373)	2023-11-04 13:59:33 -03:00
Julien Chaumond	fdcaa955e3	transformers: Add a flag to force load from safetensors (#4450)	2023-11-02 16:20:54 -03:00
oobabooga	c0655475ae	Add cache_8bit option	2023-11-02 11:23:04 -07:00
oobabooga	77abd9b69b	Add no_flash_attn option	2023-11-02 11:08:53 -07:00
oobabooga	1edf321362	Lint	2023-10-23 13:09:03 -07:00
oobabooga	df90d03e0b	Replace --mul_mat_q with --no_mul_mat_q	2023-10-22 12:23:03 -07:00
oobabooga	2d1b3332e4	Ignore warnings on Colab	2023-10-21 21:45:25 -07:00
oobabooga	506d05aede	Organize command-line arguments	2023-10-21 18:52:59 -07:00
cal066	cc632c3f33	AutoAWQ: initial support (#3999)	2023-10-05 13:19:18 -03:00
oobabooga	b6fe6acf88	Add threads_batch parameter	2023-10-01 21:28:00 -07:00
jllllll	41a2de96e5	Bump llama-cpp-python to 0.2.11	2023-10-01 18:08:10 -05:00
oobabooga	f931184b53	Increase truncation limits to 32768	2023-09-28 19:28:22 -07:00
StoyanStAtanasov	7e6ff8d1f0	Enable NUMA feature for llama_cpp_python (#4040)	2023-09-26 22:05:00 -03:00
oobabooga	1ca54faaf0	Improve --multi-user mode	2023-09-26 06:42:33 -07:00
oobabooga	7f1460af29	Change a warning	2023-09-25 20:22:27 -07:00
oobabooga	d0d221df49	Add --use_fast option (closes #3741)	2023-09-25 12:19:43 -07:00
oobabooga	00ab450c13	Multiple histories for each character (#4022)	2023-09-21 17:19:32 -03:00
oobabooga	5075087461	Fix command-line arguments being ignored	2023-09-19 13:11:46 -07:00
missionfloyd	2ad6ca8874	Add back chat buttons with --chat-buttons (#3947)	2023-09-16 00:39:37 -03:00
saltacc	f01b9aa71f	Add customizable ban tokens (#3899)	2023-09-15 18:27:27 -03:00
oobabooga	3d1c0f173d	User config precedence over GGUF metadata	2023-09-14 12:15:52 -07:00
oobabooga	2f935547c8	Minor changes	2023-09-12 15:05:21 -07:00
oobabooga	c2a309f56e	Add ExLlamaV2 and ExLlamav2_HF loaders (#3881)	2023-09-12 14:33:07 -03:00