18 Commits

Author	SHA1	Message	Date
oobabooga	32a2bbee4a	Implement auto_max_new_tokens for ExLlama	2023-08-02 11:03:56 -07:00
oobabooga	b6643e5039	Add decode functions to llama.cpp/exllama	2023-07-07 09:11:30 -07:00
oobabooga	1ba2e88551	Add truncation to exllama	2023-07-07 09:09:23 -07:00
Panchovix	10c8c197bf	Add Support for Static NTK RoPE scaling for exllama/exllama_hf (#2955)	2023-07-04 01:13:16 -03:00
ardfork	3c076c3c80	Disable half2 for ExLlama when using HIP (#2912)	2023-06-29 15:03:16 -03:00
oobabooga	79db629665	Minor bug fix	2023-06-29 13:53:06 -03:00
oobabooga	3443219cbc	Add repetition penalty range parameter to transformers (#2916)	2023-06-29 13:40:13 -03:00
oobabooga	c52290de50	ExLlama with long context (#2875)	2023-06-25 22:49:26 -03:00
jllllll	bef67af23c	Use pre-compiled python module for ExLlama (#2770)	2023-06-24 20:24:17 -03:00
oobabooga	eb30f4441f	Add ExLlama+LoRA support (#2756)	2023-06-19 12:31:24 -03:00
oobabooga	5f418f6171	Fix a memory leak (credits for the fix: Ph0rk0z)	2023-06-19 01:19:28 -03:00
Forkoz	3cae1221d4	Update exllama.py - Respect model dir parameter (#2744)	2023-06-18 13:26:30 -03:00
oobabooga	c5641b65d3	Handle leading spaces properly in ExLllama	2023-06-17 19:35:12 -03:00
oobabooga	cbd63eeeff	Fix repeated tokens with exllama	2023-06-17 19:02:08 -03:00
oobabooga	766c760cd7	Use gen_begin_reuse in exllama	2023-06-17 18:00:10 -03:00
oobabooga	b27f83c0e9	Make exllama stoppable	2023-06-16 22:03:23 -03:00
oobabooga	5f392122fd	Add gpu_split param to ExLlama ... Adapted from code created by Ph0rk0z. Thank you Ph0rk0z.	2023-06-16 20:49:36 -03:00
oobabooga	9f40032d32	Add ExLlama support (#2444)	2023-06-16 20:35:38 -03:00