17 Commits

Author	SHA1	Message	Date
oobabooga	77abd9b69b	Add no_flash_attn option	2023-11-02 11:08:53 -07:00
Brian Dashore	3345da2ea4	Add flash-attention 2 for windows (#4235)	2023-10-21 03:46:23 -03:00
Johan	1d5a015ce7	Enable special token support for exllamav2 (#4314)	2023-10-21 01:54:06 -03:00
Forkoz	8cce1f1126	Exllamav2 lora support (#4229) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2023-10-14 16:12:41 -03:00
turboderp	8a98646a21	Bump ExLlamaV2 to 0.0.5 (#4186)	2023-10-05 19:12:22 -03:00
oobabooga	56b5a4af74	exllamav2 typical_p	2023-09-28 20:10:12 -07:00
oobabooga	13ac55fa18	Reorder some functions	2023-09-19 13:51:57 -07:00
oobabooga	ff5d3d2d09	Add missing import	2023-09-18 16:26:54 -07:00
oobabooga	605ec3c9f2	Add a warning about ExLlamaV2 without flash-attn	2023-09-18 12:26:35 -07:00
oobabooga	ad8ac545a5	Tokenization improvements	2023-09-17 07:02:00 -07:00
saltacc	cd08eb0753	token probs for non HF loaders (#3957)	2023-09-17 10:42:32 -03:00
saltacc	ed6b6411fb	Fix exllama tokenizers (#3954) ... --------- Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>	2023-09-16 09:42:38 -03:00
saltacc	f01b9aa71f	Add customizable ban tokens (#3899)	2023-09-15 18:27:27 -03:00
Panchovix	34dc7306b8	Fix NTK (alpha) and RoPE scaling for exllamav2 and exllamav2_HF (#3897)	2023-09-13 02:35:09 -03:00
oobabooga	b7adf290fc	Fix ExLlama-v2 path issue	2023-09-12 17:42:22 -07:00
oobabooga	18e6b275f3	Add alpha_value/compress_pos_emb to ExLlama-v2	2023-09-12 15:02:47 -07:00
oobabooga	c2a309f56e	Add ExLlamaV2 and ExLlamav2_HF loaders (#3881)	2023-09-12 14:33:07 -03:00